[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:02:23 dist-test-slave-mjjr krb5kdc[10112](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:02:23 dist-test-slave-mjjr krb5kdc[10112](info): set up 2 sockets
Aug 11 12:02:23 dist-test-slave-mjjr krb5kdc[10112](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 12:02:24.515813 10102 mini_kdc.cc:121] Time spent starting KDC: real 1.267s user 0.001s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Aug 11 12:02:24 dist-test-slave-mjjr krb5kdc[10112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449744, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:02:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:02:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.221.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
I20260811 12:02:24.695416 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:34651
--webserver_interface=127.9.221.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:46497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:34651
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:02:25.259840 10130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:02:25.260335 10130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:02:25.260582 10130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:02:25.270087 10130 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:02:25.270287 10130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:02:25.270423 10130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:02:25.270617 10130 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:02:25.270718 10130 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:02:25.283851 10130 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:46497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:34651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:34651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.221.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:02:25.285892 10130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:02:25.287786 10130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:02:25.311621 10136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:25.311411 10138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:25.311506 10143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:02:25.315260 10130 server_base.cc:1034] running on GCE node
I20260811 12:02:25.316509 10130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:02:25.318425 10130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:02:25.319751 10130 hybrid_clock.cc:648] HybridClock initialized: now 1786449745319570 us; error 93 us; skew 500 ppm
I20260811 12:02:25.320286 10130 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:02:25 dist-test-slave-mjjr krb5kdc[10112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449745, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:02:25.324776 10130 init.cc:377] Logged in from keytab as kudu/127.9.221.190@KRBTEST.COM (short username kudu)
I20260811 12:02:25.327800 10130 webserver.cc:460] Webserver started at http://127.9.221.190:36747/ using document root <none> and password file <none>
I20260811 12:02:25.328948 10130 fs_manager.cc:359] Metadata directory not provided
I20260811 12:02:25.329128 10130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:02:25.329627 10130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:02:25.335244 10130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "dee5aa6868334317855ff4c4c3ca4b6b"
format_stamp: "Formatted at 2026-08-11 12:02:25 on dist-test-slave-mjjr"
server_key: "04783e25fed21fc3ce80ae8dee0df1dc"
server_key_iv: "0b6eab84c44fa281a25f341742e50096"
server_key_version: "encryptionkey@0"
I20260811 12:02:25.336627 10130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "dee5aa6868334317855ff4c4c3ca4b6b"
format_stamp: "Formatted at 2026-08-11 12:02:25 on dist-test-slave-mjjr"
server_key: "04783e25fed21fc3ce80ae8dee0df1dc"
server_key_iv: "0b6eab84c44fa281a25f341742e50096"
server_key_version: "encryptionkey@0"
I20260811 12:02:25.345372 10130 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.002s sys 0.006s
I20260811 12:02:25.351627 10154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:25.353691 10130 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.002s
I20260811 12:02:25.354055 10130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "dee5aa6868334317855ff4c4c3ca4b6b"
format_stamp: "Formatted at 2026-08-11 12:02:25 on dist-test-slave-mjjr"
server_key: "04783e25fed21fc3ce80ae8dee0df1dc"
server_key_iv: "0b6eab84c44fa281a25f341742e50096"
server_key_version: "encryptionkey@0"
I20260811 12:02:25.354394 10130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:02:25.377828 10130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:02:25.384009 10130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:02:25.384522 10130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:02:25.425536 10130 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.190:34651
I20260811 12:02:25.425631 10233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.190:34651 every 8 connection(s)
I20260811 12:02:25.427910 10130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:02:25.430955 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 10130
I20260811 12:02:25.431464 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:02:25.435560 10102 external_mini_cluster.cc:1428] Setting key 2e52140fd4f835e9e4aa84a7c427dbf6
I20260811 12:02:25.437551 10236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:02:25.465240 10236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dee5aa6868334317855ff4c4c3ca4b6b: Bootstrap starting.
I20260811 12:02:25.474069 10236 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dee5aa6868334317855ff4c4c3ca4b6b: Neither blocks nor log segments found. Creating new log.
I20260811 12:02:25.476531 10236 log.cc:824] T 00000000000000000000000000000000 P dee5aa6868334317855ff4c4c3ca4b6b: Log is configured to *not* fsync() on all Append() calls
I20260811 12:02:25.481681 10236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dee5aa6868334317855ff4c4c3ca4b6b: No bootstrap required, opened a new log
Aug 11 12:02:25 dist-test-slave-mjjr krb5kdc[10112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449744, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:02:25.495922 10236 raft_consensus.cc:348] T 00000000000000000000000000000000 P dee5aa6868334317855ff4c4c3ca4b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dee5aa6868334317855ff4c4c3ca4b6b" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 34651 } }
I20260811 12:02:25.496583 10236 raft_consensus.cc:374] T 00000000000000000000000000000000 P dee5aa6868334317855ff4c4c3ca4b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:02:25.496865 10236 raft_consensus.cc:729] T 00000000000000000000000000000000 P dee5aa6868334317855ff4c4c3ca4b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dee5aa6868334317855ff4c4c3ca4b6b, State: Initialized, Role: FOLLOWER
I20260811 12:02:25.498111 10236 consensus_queue.cc:260] T 00000000000000000000000000000000 P dee5aa6868334317855ff4c4c3ca4b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dee5aa6868334317855ff4c4c3ca4b6b" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 34651 } }
I20260811 12:02:25.498745 10236 raft_consensus.cc:388] T 00000000000000000000000000000000 P dee5aa6868334317855ff4c4c3ca4b6b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:02:25.499048 10236 raft_consensus.cc:482] T 00000000000000000000000000000000 P dee5aa6868334317855ff4c4c3ca4b6b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:02:25.499374 10236 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dee5aa6868334317855ff4c4c3ca4b6b [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:02:25.501965 10243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:25.445561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60392 (local address 127.9.221.190:34651)
0811 12:02:25.446962 (+ 1401us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:25.446973 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:25.447020 (+ 47us) server_negotiation.cc:407] Connection header received
0811 12:02:25.465197 (+ 18177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:25.465219 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:25.465809 (+ 590us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:25.466162 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:25.473254 (+ 7092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:25.475362 (+ 2108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:25.478712 (+ 3350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:25.479502 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:25.479770 (+ 268us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:25.490188 (+ 10418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:25.490222 (+ 34us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:25.490257 (+ 35us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:25.490320 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:25.494853 (+ 4533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:25.495783 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:25.495791 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:25.495800 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:25.496035 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:25.496740 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:25.496748 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:25.496757 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:25.497800 (+ 1043us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:25.498308 (+ 508us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:25.499399 (+ 1091us) server_negotiation.cc:299] Negotiation successful
0811 12:02:25.500012 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1051,"thread_start_us":662,"threads_started":1}
I20260811 12:02:25.503162 10236 raft_consensus.cc:504] T 00000000000000000000000000000000 P dee5aa6868334317855ff4c4c3ca4b6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dee5aa6868334317855ff4c4c3ca4b6b" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 34651 } }
I20260811 12:02:25.504163 10236 leader_election.cc:302] T 00000000000000000000000000000000 P dee5aa6868334317855ff4c4c3ca4b6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dee5aa6868334317855ff4c4c3ca4b6b; no voters:
I20260811 12:02:25.505317 10236 leader_election.cc:288] T 00000000000000000000000000000000 P dee5aa6868334317855ff4c4c3ca4b6b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:02:25.505795 10246 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dee5aa6868334317855ff4c4c3ca4b6b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:02:25.508152 10246 raft_consensus.cc:686] T 00000000000000000000000000000000 P dee5aa6868334317855ff4c4c3ca4b6b [term 1 LEADER]: Becoming Leader. State: Replica: dee5aa6868334317855ff4c4c3ca4b6b, State: Running, Role: LEADER
I20260811 12:02:25.509135 10246 consensus_queue.cc:237] T 00000000000000000000000000000000 P dee5aa6868334317855ff4c4c3ca4b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dee5aa6868334317855ff4c4c3ca4b6b" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 34651 } }
I20260811 12:02:25.509615 10236 sys_catalog.cc:564] T 00000000000000000000000000000000 P dee5aa6868334317855ff4c4c3ca4b6b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:02:25.515281 10252 sys_catalog.cc:455] T 00000000000000000000000000000000 P dee5aa6868334317855ff4c4c3ca4b6b [sys.catalog]: SysCatalogTable state changed. Reason: New leader dee5aa6868334317855ff4c4c3ca4b6b. Latest consensus state: current_term: 1 leader_uuid: "dee5aa6868334317855ff4c4c3ca4b6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dee5aa6868334317855ff4c4c3ca4b6b" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 34651 } } }
I20260811 12:02:25.516815 10252 sys_catalog.cc:458] T 00000000000000000000000000000000 P dee5aa6868334317855ff4c4c3ca4b6b [sys.catalog]: This master's current role is: LEADER
I20260811 12:02:25.516644 10247 sys_catalog.cc:455] T 00000000000000000000000000000000 P dee5aa6868334317855ff4c4c3ca4b6b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dee5aa6868334317855ff4c4c3ca4b6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dee5aa6868334317855ff4c4c3ca4b6b" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 34651 } } }
I20260811 12:02:25.518254 10247 sys_catalog.cc:458] T 00000000000000000000000000000000 P dee5aa6868334317855ff4c4c3ca4b6b [sys.catalog]: This master's current role is: LEADER
I20260811 12:02:25.525669 10264 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:02:25.535029 10264 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:02:25.551129 10264 catalog_manager.cc:1347] Generated new cluster ID: eb85fa39c83d4ac7a0e3642f3d1c0a20
I20260811 12:02:25.551343 10264 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:02:25.569882 10264 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:02:25.571849 10264 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:02:25.580600 10264 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dee5aa6868334317855ff4c4c3ca4b6b: Generated new TSK 0
I20260811 12:02:25.581694 10264 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.221.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab.
Entry for principal kudu/127.9.221.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab.
I20260811 12:02:25.635293 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.129:0
--local_ip_for_outbound_sockets=127.9.221.129
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:34651
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab
--principal=kudu/127.9.221.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:46497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:02:26.205050 10281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:02:26.205467 10281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:02:26.205721 10281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:02:26.215706 10281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:02:26.215906 10281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:02:26.216120 10281 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.129
I20260811 12:02:26.229429 10281 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:46497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab
--principal=kudu/127.9.221.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:34651
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.129
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:02:26.231137 10281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:02:26.232749 10281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:02:26.249776 10286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:26.249724 10288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:26.249810 10291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:02:26.251545 10281 server_base.cc:1034] running on GCE node
I20260811 12:02:26.252213 10281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:02:26.253348 10281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:02:26.254592 10281 hybrid_clock.cc:648] HybridClock initialized: now 1786449746254514 us; error 40 us; skew 500 ppm
I20260811 12:02:26.255024 10281 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:02:26 dist-test-slave-mjjr krb5kdc[10112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449746, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:02:26.259334 10281 init.cc:377] Logged in from keytab as kudu/127.9.221.129@KRBTEST.COM (short username kudu)
I20260811 12:02:26.261150 10281 webserver.cc:460] Webserver started at http://127.9.221.129:40423/ using document root <none> and password file <none>
I20260811 12:02:26.262043 10281 fs_manager.cc:359] Metadata directory not provided
I20260811 12:02:26.262352 10281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:02:26.262789 10281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:02:26.266438 10281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-0/data/instance:
uuid: "3f2d93e6771f4cb9861e433c9c813dcf"
format_stamp: "Formatted at 2026-08-11 12:02:26 on dist-test-slave-mjjr"
server_key: "3af1c2b748b525cd7ff653adb4d4d657"
server_key_iv: "aca62ec2388c71c024f974d365208755"
server_key_version: "encryptionkey@0"
I20260811 12:02:26.267349 10281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance:
uuid: "3f2d93e6771f4cb9861e433c9c813dcf"
format_stamp: "Formatted at 2026-08-11 12:02:26 on dist-test-slave-mjjr"
server_key: "3af1c2b748b525cd7ff653adb4d4d657"
server_key_iv: "aca62ec2388c71c024f974d365208755"
server_key_version: "encryptionkey@0"
I20260811 12:02:26.273766 10281 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 12:02:26.278229 10300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:26.279722 10281 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:02:26.279924 10281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-0/wal
uuid: "3f2d93e6771f4cb9861e433c9c813dcf"
format_stamp: "Formatted at 2026-08-11 12:02:26 on dist-test-slave-mjjr"
server_key: "3af1c2b748b525cd7ff653adb4d4d657"
server_key_iv: "aca62ec2388c71c024f974d365208755"
server_key_version: "encryptionkey@0"
I20260811 12:02:26.280203 10281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:02:26.297672 10281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:02:26.302767 10281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:02:26.303212 10281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:02:26.306232 10281 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:02:26.307994 10281 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:02:26.308096 10281 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:26.308310 10281 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:02:26.308434 10281 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:26.390496 10281 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.129:40583
I20260811 12:02:26.390764 10482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.129:40583 every 8 connection(s)
I20260811 12:02:26.392127 10281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
I20260811 12:02:26.401750 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 10281
I20260811 12:02:26.402055 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance
I20260811 12:02:26.402722 10102 external_mini_cluster.cc:1428] Setting key 10dbe89d629f0fe755dc79879efefc7d
Aug 11 12:02:26 dist-test-slave-mjjr krb5kdc[10112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449746, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:02:26.422843 10488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:26.396136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:40629 (local address 127.9.221.190:34651)
0811 12:02:26.396994 (+ 858us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:26.396999 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:26.398174 (+ 1175us) server_negotiation.cc:407] Connection header received
0811 12:02:26.400742 (+ 2568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:26.400748 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:26.400875 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:26.401011 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:26.403039 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:26.404246 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:26.407350 (+ 3104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:26.407956 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:26.408066 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:26.413451 (+ 5385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:26.413460 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:26.413469 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:26.413519 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:26.418285 (+ 4766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:26.419149 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:26.419157 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:26.419164 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:26.419389 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:26.420101 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:26.420108 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:26.420114 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:26.420667 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:26.420914 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:26.421814 (+ 900us) server_negotiation.cc:299] Negotiation successful
0811 12:02:26.422132 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":716,"thread_start_us":523,"threads_started":1}
I20260811 12:02:26.423189 10489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:26.397033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34651 (local address 127.9.221.129:40629)
0811 12:02:26.397994 (+ 961us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:26.398058 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:26.400477 (+ 2419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:26.401337 (+ 860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:26.401346 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:26.402055 (+ 709us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:02:26.402824 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:26.402869 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:26.404394 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:26.404399 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:26.407172 (+ 2773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:26.407192 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:26.408254 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:26.408264 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:26.408729 (+ 465us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:26.409660 (+ 931us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:02:26.409721 (+ 61us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:02:26.413182 (+ 3461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:02:26.418446 (+ 5264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:26.418453 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:26.418468 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:26.419008 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:26.419565 (+ 557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:26.419568 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:26.419571 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:26.419957 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:26.420855 (+ 898us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:02:26.420868 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:02:26.421299 (+ 431us) client_negotiation.cc:769] Sending connection context
0811 12:02:26.421581 (+ 282us) client_negotiation.cc:240] Negotiation successful
0811 12:02:26.422000 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":687,"thread_start_us":435,"threads_started":1}
I20260811 12:02:26.425232 10483 heartbeater.cc:347] Connected to a master server at 127.9.221.190:34651
I20260811 12:02:26.425712 10483 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.9.221.130@KRBTEST.COM; defaulting to no policy
I20260811 12:02:26.426959 10483 heartbeater.cc:511] Master 127.9.221.190:34651 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.130@KRBTEST.COM" created.
I20260811 12:02:26.431268 10183 ts_manager.cc:194] Registered new tserver with Master: 3f2d93e6771f4cb9861e433c9c813dcf (127.9.221.129:40583)
I20260811 12:02:26.441617 10183 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.129@KRBTEST.COM'} at 127.9.221.129:40629
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab.
Entry for principal kudu/127.9.221.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab.
I20260811 12:02:26.463143 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.130:0
--local_ip_for_outbound_sockets=127.9.221.130
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:34651
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab
--principal=kudu/127.9.221.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:46497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:02:27.041332 10494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:02:27.041659 10494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:02:27.041848 10494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:02:27.051210 10494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:02:27.051373 10494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:02:27.051563 10494 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.130
I20260811 12:02:27.063313 10494 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:46497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab
--principal=kudu/127.9.221.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:34651
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.130
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:02:27.064749 10494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:02:27.066244 10494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:02:27.082239 10501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:27.082615 10502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:27.082966 10504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:02:27.084545 10494 server_base.cc:1034] running on GCE node
I20260811 12:02:27.085378 10494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:02:27.086535 10494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:02:27.087797 10494 hybrid_clock.cc:648] HybridClock initialized: now 1786449747087682 us; error 42 us; skew 500 ppm
I20260811 12:02:27.088186 10494 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:02:27 dist-test-slave-mjjr krb5kdc[10112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449747, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:02:27.092181 10494 init.cc:377] Logged in from keytab as kudu/127.9.221.130@KRBTEST.COM (short username kudu)
I20260811 12:02:27.094126 10494 webserver.cc:460] Webserver started at http://127.9.221.130:42059/ using document root <none> and password file <none>
I20260811 12:02:27.094908 10494 fs_manager.cc:359] Metadata directory not provided
I20260811 12:02:27.095096 10494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:02:27.095461 10494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:02:27.098969 10494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-1/data/instance:
uuid: "3a4f3882c69d4caf8cb2f626853c1aef"
format_stamp: "Formatted at 2026-08-11 12:02:27 on dist-test-slave-mjjr"
server_key: "959f41fae0f74c6722c479441de1499d"
server_key_iv: "222a98bee5cf7d5a95d35ef1a96c54c0"
server_key_version: "encryptionkey@0"
I20260811 12:02:27.099956 10494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance:
uuid: "3a4f3882c69d4caf8cb2f626853c1aef"
format_stamp: "Formatted at 2026-08-11 12:02:27 on dist-test-slave-mjjr"
server_key: "959f41fae0f74c6722c479441de1499d"
server_key_iv: "222a98bee5cf7d5a95d35ef1a96c54c0"
server_key_version: "encryptionkey@0"
I20260811 12:02:27.106117 10494 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.005s
I20260811 12:02:27.110689 10517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:27.112406 10494 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260811 12:02:27.112658 10494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-1/wal
uuid: "3a4f3882c69d4caf8cb2f626853c1aef"
format_stamp: "Formatted at 2026-08-11 12:02:27 on dist-test-slave-mjjr"
server_key: "959f41fae0f74c6722c479441de1499d"
server_key_iv: "222a98bee5cf7d5a95d35ef1a96c54c0"
server_key_version: "encryptionkey@0"
I20260811 12:02:27.112962 10494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:02:27.144384 10494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:02:27.150789 10494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:02:27.151304 10494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:02:27.152870 10494 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:02:27.154759 10494 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:02:27.154937 10494 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:27.155179 10494 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:02:27.155319 10494 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:27.229926 10494 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.130:45089
I20260811 12:02:27.230049 10718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.130:45089 every 8 connection(s)
I20260811 12:02:27.231518 10494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
I20260811 12:02:27.237810 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 10494
I20260811 12:02:27.238054 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance
I20260811 12:02:27.238502 10102 external_mini_cluster.cc:1428] Setting key bfb56bd0cadd664d08ee536e37cb63b7
Aug 11 12:02:27 dist-test-slave-mjjr krb5kdc[10112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449747, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.130@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.221.131@KRBTEST.COM; defaulting to no policy
I20260811 12:02:27.260882 10727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:27.234923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:34921 (local address 127.9.221.190:34651)
0811 12:02:27.235711 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:27.235719 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:27.236976 (+ 1257us) server_negotiation.cc:407] Connection header received
0811 12:02:27.239421 (+ 2445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:27.239428 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:27.239532 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:27.239698 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:27.242338 (+ 2640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:27.243194 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:27.245454 (+ 2260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:27.246621 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:27.246801 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:27.251203 (+ 4402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:27.251213 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:27.251221 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:27.251273 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:27.256777 (+ 5504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:27.257599 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:27.257608 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:27.257615 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:27.257851 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:27.258414 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:27.258420 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:27.258425 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:27.258923 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:27.259181 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:27.260007 (+ 826us) server_negotiation.cc:299] Negotiation successful
0811 12:02:27.260190 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":498,"threads_started":1}
I20260811 12:02:27.261924 10728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:27.235622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34651 (local address 127.9.221.130:34921)
0811 12:02:27.236820 (+ 1198us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:27.236857 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:27.239107 (+ 2250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:27.240016 (+ 909us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:27.240026 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:27.241149 (+ 1123us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:02:27.242131 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:27.242150 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:27.243385 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:27.243390 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:27.245198 (+ 1808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:27.245213 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:27.246958 (+ 1745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:27.246965 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:27.247216 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:27.248080 (+ 864us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:02:27.248107 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:02:27.250869 (+ 2762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:02:27.256934 (+ 6065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:27.256941 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:27.256952 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:27.257412 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:27.257993 (+ 581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:27.257996 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:27.257998 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:27.258260 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:27.259111 (+ 851us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:02:27.259117 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:02:27.259437 (+ 320us) client_negotiation.cc:769] Sending connection context
0811 12:02:27.259799 (+ 362us) client_negotiation.cc:240] Negotiation successful
0811 12:02:27.260275 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":878,"thread_start_us":582,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.131@KRBTEST.COM" created.
I20260811 12:02:27.263499 10722 heartbeater.cc:347] Connected to a master server at 127.9.221.190:34651
I20260811 12:02:27.263860 10722 heartbeater.cc:464] Registering TS with master...
I20260811 12:02:27.264770 10722 heartbeater.cc:511] Master 127.9.221.190:34651 requested a full tablet report, sending...
I20260811 12:02:27.266435 10183 ts_manager.cc:194] Registered new tserver with Master: 3a4f3882c69d4caf8cb2f626853c1aef (127.9.221.130:45089)
I20260811 12:02:27.267401 10183 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.130@KRBTEST.COM'} at 127.9.221.130:34921
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab.
Entry for principal kudu/127.9.221.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab.
I20260811 12:02:27.286674 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.131:0
--local_ip_for_outbound_sockets=127.9.221.131
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:34651
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab
--principal=kudu/127.9.221.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:46497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:02:27.447765 10483 heartbeater.cc:503] Master 127.9.221.190:34651 was elected leader, sending a full tablet report...
W20260811 12:02:27.855623 10734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:02:27.856110 10734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:02:27.856348 10734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:02:27.867815 10734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:02:27.868028 10734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:02:27.868327 10734 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.131
I20260811 12:02:27.881759 10734 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:46497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab
--principal=kudu/127.9.221.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:34651
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.131
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:02:27.883500 10734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:02:27.885300 10734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:02:27.903903 10741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:27.904105 10742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:27.904927 10748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:02:27.906227 10734 server_base.cc:1034] running on GCE node
I20260811 12:02:27.906843 10734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:02:27.907799 10734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:02:27.909029 10734 hybrid_clock.cc:648] HybridClock initialized: now 1786449747908956 us; error 52 us; skew 500 ppm
I20260811 12:02:27.909381 10734 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:02:27 dist-test-slave-mjjr krb5kdc[10112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449747, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:02:27.913410 10734 init.cc:377] Logged in from keytab as kudu/127.9.221.131@KRBTEST.COM (short username kudu)
I20260811 12:02:27.915182 10734 webserver.cc:460] Webserver started at http://127.9.221.131:45053/ using document root <none> and password file <none>
I20260811 12:02:27.915979 10734 fs_manager.cc:359] Metadata directory not provided
I20260811 12:02:27.916124 10734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:02:27.916476 10734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:02:27.919787 10734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-2/data/instance:
uuid: "799b2a1f6ef54c8c9db1e0b4e2d2a300"
format_stamp: "Formatted at 2026-08-11 12:02:27 on dist-test-slave-mjjr"
server_key: "78827017ca6878d33e30f07a3aa505ef"
server_key_iv: "ccdce1c215e090b34f7edd6c0349530c"
server_key_version: "encryptionkey@0"
I20260811 12:02:27.920723 10734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance:
uuid: "799b2a1f6ef54c8c9db1e0b4e2d2a300"
format_stamp: "Formatted at 2026-08-11 12:02:27 on dist-test-slave-mjjr"
server_key: "78827017ca6878d33e30f07a3aa505ef"
server_key_iv: "ccdce1c215e090b34f7edd6c0349530c"
server_key_version: "encryptionkey@0"
I20260811 12:02:27.926900 10734 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260811 12:02:27.931401 10757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:27.932899 10734 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 12:02:27.933122 10734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-2/wal
uuid: "799b2a1f6ef54c8c9db1e0b4e2d2a300"
format_stamp: "Formatted at 2026-08-11 12:02:27 on dist-test-slave-mjjr"
server_key: "78827017ca6878d33e30f07a3aa505ef"
server_key_iv: "ccdce1c215e090b34f7edd6c0349530c"
server_key_version: "encryptionkey@0"
I20260811 12:02:27.933411 10734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:02:27.957909 10734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:02:27.963469 10734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:02:27.963979 10734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:02:27.965413 10734 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:02:27.967211 10734 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:02:27.967396 10734 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:27.967626 10734 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:02:27.967782 10734 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260811 12:02:28.045243 10734 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.131:40001
I20260811 12:02:28.045421 10937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.131:40001 every 8 connection(s)
I20260811 12:02:28.046833 10734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
I20260811 12:02:28.049250 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 10734
I20260811 12:02:28.049526 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance
I20260811 12:02:28.050026 10102 external_mini_cluster.cc:1428] Setting key 52a85a3de04252f9141ada50108f2fc5
Aug 11 12:02:28 dist-test-slave-mjjr krb5kdc[10112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449747, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.131@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:02:28.074753 10941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:28.050841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:44627 (local address 127.9.221.190:34651)
0811 12:02:28.051597 (+ 756us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:28.051604 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:28.053332 (+ 1728us) server_negotiation.cc:407] Connection header received
0811 12:02:28.056828 (+ 3496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:28.056834 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:28.056988 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:28.057123 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:28.058984 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:28.059978 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:28.062014 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:28.062682 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:28.062843 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:28.068106 (+ 5263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:28.068111 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:28.068115 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:28.068181 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:28.071560 (+ 3379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:28.072400 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:28.072403 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:28.072406 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:28.072562 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:28.073015 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:28.073018 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:28.073020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:28.073366 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:28.073489 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:28.074171 (+ 682us) server_negotiation.cc:299] Negotiation successful
0811 12:02:28.074369 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":424,"threads_started":1}
I20260811 12:02:28.075871 10942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:28.051656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34651 (local address 127.9.221.131:44627)
0811 12:02:28.053141 (+ 1485us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:28.053178 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:28.056515 (+ 3337us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:28.057406 (+ 891us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:28.057414 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:28.058058 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:02:28.058796 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:28.058815 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:28.060203 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:28.060207 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:28.061815 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:28.061829 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:28.063013 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:28.063023 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:28.063328 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:28.064348 (+ 1020us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:02:28.064372 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:02:28.067776 (+ 3404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:02:28.071729 (+ 3953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:28.071738 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:28.071758 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:28.072289 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:28.072657 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:28.072661 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:28.072663 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:28.072911 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:28.073481 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:02:28.073486 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:02:28.073831 (+ 345us) client_negotiation.cc:769] Sending connection context
0811 12:02:28.074077 (+ 246us) client_negotiation.cc:240] Negotiation successful
0811 12:02:28.074500 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1140,"thread_start_us":727,"threads_started":1}
I20260811 12:02:28.077416 10939 heartbeater.cc:347] Connected to a master server at 127.9.221.190:34651
I20260811 12:02:28.077736 10939 heartbeater.cc:464] Registering TS with master...
I20260811 12:02:28.078709 10939 heartbeater.cc:511] Master 127.9.221.190:34651 requested a full tablet report, sending...
I20260811 12:02:28.080816 10183 ts_manager.cc:194] Registered new tserver with Master: 799b2a1f6ef54c8c9db1e0b4e2d2a300 (127.9.221.131:40001)
I20260811 12:02:28.081537 10102 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:02:28.081859 10183 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.131@KRBTEST.COM'} at 127.9.221.131:44627
Aug 11 12:02:28 dist-test-slave-mjjr krb5kdc[10112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449748, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 12:02:28 dist-test-slave-mjjr krb5kdc[10112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449748, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.221.129@KRBTEST.COM
I20260811 12:02:28.136627 10948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:28.116023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53194 (local address 127.9.221.129:40583)
0811 12:02:28.117172 (+ 1149us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:28.117186 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:28.117305 (+ 119us) server_negotiation.cc:407] Connection header received
0811 12:02:28.117589 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:28.117599 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:28.117862 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:28.118049 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:28.120638 (+ 2589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:28.122035 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:28.123821 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:28.124694 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:28.124854 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:28.129693 (+ 4839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:28.129704 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:28.129718 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:28.129757 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:28.133201 (+ 3444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:28.133892 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:28.133899 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:28.133901 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:28.134076 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:28.134481 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:28.134484 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:28.134486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:28.134839 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:28.135227 (+ 388us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:28.135770 (+ 543us) server_negotiation.cc:299] Negotiation successful
0811 12:02:28.136162 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":946,"thread_start_us":644,"threads_started":1}
W20260811 12:02:28.138136 10345 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53194
Aug 11 12:02:28 dist-test-slave-mjjr krb5kdc[10112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 12:02:28 dist-test-slave-mjjr krb5kdc[10112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.129@KRBTEST.COM
I20260811 12:02:28.190436 10948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:28.174821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53198 (local address 127.9.221.129:40583)
0811 12:02:28.175066 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:28.175070 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:28.175154 (+ 84us) server_negotiation.cc:407] Connection header received
0811 12:02:28.175329 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:28.175334 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:28.175415 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:28.175505 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:28.177222 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:28.177907 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:28.179201 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:28.179592 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:28.179774 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:28.184341 (+ 4567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:28.184346 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:28.184353 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:28.184386 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:28.187831 (+ 3445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:28.188505 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:28.188508 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:28.188510 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:28.188694 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:28.189021 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:28.189024 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:28.189026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:28.189307 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:28.189541 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:28.189830 (+ 289us) server_negotiation.cc:299] Negotiation successful
0811 12:02:28.190006 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260811 12:02:28.192339 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 10281
I20260811 12:02:28.207361 10948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:28.193351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53204 (local address 127.9.221.129:40583)
0811 12:02:28.193515 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:28.193519 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:28.193857 (+ 338us) server_negotiation.cc:407] Connection header received
0811 12:02:28.193965 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:28.193969 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:28.194017 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:28.194090 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:28.195287 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:28.196244 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:28.197776 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:28.198287 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:28.198381 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:28.201302 (+ 2921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:28.201306 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:28.201309 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:28.201337 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:28.204785 (+ 3448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:28.205478 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:28.205482 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:28.205484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:28.205655 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:28.206022 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:28.206025 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:28.206027 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:28.206330 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:28.206550 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:28.206771 (+ 221us) server_negotiation.cc:299] Negotiation successful
0811 12:02:28.206932 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 12:02:28.208196 10477 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:02:28.270509 10722 heartbeater.cc:503] Master 127.9.221.190:34651 was elected leader, sending a full tablet report...
I20260811 12:02:28.300581 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 10281
I20260811 12:02:28.325686 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 10494
Aug 11 12:02:28 dist-test-slave-mjjr krb5kdc[10112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.130@KRBTEST.COM
I20260811 12:02:28.347354 10957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:28.326839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47588 (local address 127.9.221.130:45089)
0811 12:02:28.327442 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:28.327449 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:28.327475 (+ 26us) server_negotiation.cc:407] Connection header received
0811 12:02:28.327534 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:28.327540 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:28.327838 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:28.328004 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:28.329211 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:28.330180 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:28.331373 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:28.332039 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:28.332177 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:28.341095 (+ 8918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:28.341107 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:28.341120 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:28.341154 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:28.344063 (+ 2909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:28.344728 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:28.344738 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:28.344745 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:28.344885 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:28.345441 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:28.345447 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:28.345452 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:28.345872 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:28.346208 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:28.346626 (+ 418us) server_negotiation.cc:299] Negotiation successful
0811 12:02:28.346920 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":294,"threads_started":1}
I20260811 12:02:28.349072 10712 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:02:28.432389 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 10494
I20260811 12:02:28.457001 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 10734
Aug 11 12:02:28 dist-test-slave-mjjr krb5kdc[10112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.131@KRBTEST.COM
I20260811 12:02:28.472990 10960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:28.458181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44908 (local address 127.9.221.131:40001)
0811 12:02:28.458730 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:28.458738 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:28.458767 (+ 29us) server_negotiation.cc:407] Connection header received
0811 12:02:28.458836 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:28.458843 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:28.459100 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:28.459270 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:28.460539 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:28.461588 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:28.462800 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:28.463331 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:28.463423 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:28.467609 (+ 4186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:28.467619 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:28.467632 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:28.467709 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:28.470577 (+ 2868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:28.471152 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:28.471159 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:28.471161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:28.471284 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:28.471606 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:28.471609 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:28.471611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:28.472044 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:28.472269 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:28.472503 (+ 234us) server_negotiation.cc:299] Negotiation successful
0811 12:02:28.472694 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":275,"threads_started":1}
I20260811 12:02:28.474246 10931 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:02:28.557163 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 10734
I20260811 12:02:28.582482 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 10130
I20260811 12:02:28.583724 10229 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:02:28.686786 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 10130
2026-08-11T12:02:28Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (5504 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:02:28 dist-test-slave-mjjr krb5kdc[10968](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:02:28 dist-test-slave-mjjr krb5kdc[10968](info): set up 2 sockets
Aug 11 12:02:28 dist-test-slave-mjjr krb5kdc[10968](info): commencing operation
krb5kdc: starting...
W20260811 12:02:29.999043 10102 mini_kdc.cc:121] Time spent starting KDC: real 1.261s user 0.000s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Aug 11 12:02:30 dist-test-slave-mjjr krb5kdc[10968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449750, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:02:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:02:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.221.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
I20260811 12:02:30.174824 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:44099
--webserver_interface=127.9.221.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:42927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:44099
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:02:30.498086 10986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:02:30.498430 10986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:02:30.498610 10986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:02:30.505995 10986 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:02:30.506134 10986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:02:30.506239 10986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:02:30.506345 10986 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:02:30.506455 10986 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:02:30.516331 10986 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:42927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:44099
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:44099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.221.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:02:30.517702 10986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:02:30.519099 10986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:02:30.531396 10996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:30.531353 11001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:30.531629 10997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:02:30.532261 10986 server_base.cc:1034] running on GCE node
I20260811 12:02:30.533004 10986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:02:30.534337 10986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:02:30.535574 10986 hybrid_clock.cc:648] HybridClock initialized: now 1786449750535533 us; error 46 us; skew 500 ppm
I20260811 12:02:30.535938 10986 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:02:30 dist-test-slave-mjjr krb5kdc[10968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449750, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:02:30.539155 10986 init.cc:377] Logged in from keytab as kudu/127.9.221.190@KRBTEST.COM (short username kudu)
I20260811 12:02:30.540661 10986 webserver.cc:460] Webserver started at http://127.9.221.190:35761/ using document root <none> and password file <none>
I20260811 12:02:30.541338 10986 fs_manager.cc:359] Metadata directory not provided
I20260811 12:02:30.541495 10986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:02:30.541819 10986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:02:30.544816 10986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "c2fc2c133c7c4046990cda952548edf7"
format_stamp: "Formatted at 2026-08-11 12:02:30 on dist-test-slave-mjjr"
server_key: "542ada1a26e517646cbe504b54697f9a"
server_key_iv: "8398036926a84c46cbd9b80297f1bac2"
server_key_version: "encryptionkey@0"
I20260811 12:02:30.545575 10986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "c2fc2c133c7c4046990cda952548edf7"
format_stamp: "Formatted at 2026-08-11 12:02:30 on dist-test-slave-mjjr"
server_key: "542ada1a26e517646cbe504b54697f9a"
server_key_iv: "8398036926a84c46cbd9b80297f1bac2"
server_key_version: "encryptionkey@0"
I20260811 12:02:30.550557 10986 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 12:02:30.553895 11015 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:30.555126 10986 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 12:02:30.555313 10986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "c2fc2c133c7c4046990cda952548edf7"
format_stamp: "Formatted at 2026-08-11 12:02:30 on dist-test-slave-mjjr"
server_key: "542ada1a26e517646cbe504b54697f9a"
server_key_iv: "8398036926a84c46cbd9b80297f1bac2"
server_key_version: "encryptionkey@0"
I20260811 12:02:30.555516 10986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:02:30.579085 10986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:02:30.583267 10986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:02:30.583582 10986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:02:30.602296 10986 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.190:44099
I20260811 12:02:30.602322 11099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.190:44099 every 8 connection(s)
I20260811 12:02:30.603746 10986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:02:30.607506 11100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:02:30.610289 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 10986
I20260811 12:02:30.610529 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:02:30.611348 10102 external_mini_cluster.cc:1428] Setting key 7e00f0300ccf3d4e46947a617e4355b0
I20260811 12:02:30.620226 11100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2fc2c133c7c4046990cda952548edf7: Bootstrap starting.
I20260811 12:02:30.623977 11100 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c2fc2c133c7c4046990cda952548edf7: Neither blocks nor log segments found. Creating new log.
I20260811 12:02:30.625306 11100 log.cc:824] T 00000000000000000000000000000000 P c2fc2c133c7c4046990cda952548edf7: Log is configured to *not* fsync() on all Append() calls
Aug 11 12:02:30 dist-test-slave-mjjr krb5kdc[10968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449750, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:02:30.628659 11100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2fc2c133c7c4046990cda952548edf7: No bootstrap required, opened a new log
I20260811 12:02:30.635210 11103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:30.613607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56580 (local address 127.9.221.190:44099)
0811 12:02:30.614402 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:30.614413 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:30.614456 (+ 43us) server_negotiation.cc:407] Connection header received
0811 12:02:30.616192 (+ 1736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:30.616214 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:30.616600 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:30.616917 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:30.618716 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:30.620086 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:30.621792 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:30.622468 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:30.622604 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:30.627824 (+ 5220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:30.627832 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:30.627845 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:30.627878 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:30.631036 (+ 3158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:30.631893 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:30.631900 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:30.631905 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:30.632078 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:30.632479 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:30.632482 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:30.632484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:30.632996 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:30.633205 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:30.633766 (+ 561us) server_negotiation.cc:299] Negotiation successful
0811 12:02:30.634154 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":395,"threads_started":1}
I20260811 12:02:30.637985 11100 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2fc2c133c7c4046990cda952548edf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2fc2c133c7c4046990cda952548edf7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 44099 } }
I20260811 12:02:30.638448 11100 raft_consensus.cc:374] T 00000000000000000000000000000000 P c2fc2c133c7c4046990cda952548edf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:02:30.638553 11100 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2fc2c133c7c4046990cda952548edf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2fc2c133c7c4046990cda952548edf7, State: Initialized, Role: FOLLOWER
I20260811 12:02:30.639142 11100 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2fc2c133c7c4046990cda952548edf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2fc2c133c7c4046990cda952548edf7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 44099 } }
I20260811 12:02:30.639376 11100 raft_consensus.cc:388] T 00000000000000000000000000000000 P c2fc2c133c7c4046990cda952548edf7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:02:30.639529 11100 raft_consensus.cc:482] T 00000000000000000000000000000000 P c2fc2c133c7c4046990cda952548edf7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:02:30.639719 11100 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2fc2c133c7c4046990cda952548edf7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:02:30.642129 11100 raft_consensus.cc:504] T 00000000000000000000000000000000 P c2fc2c133c7c4046990cda952548edf7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2fc2c133c7c4046990cda952548edf7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 44099 } }
I20260811 12:02:30.642633 11100 leader_election.cc:302] T 00000000000000000000000000000000 P c2fc2c133c7c4046990cda952548edf7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c2fc2c133c7c4046990cda952548edf7; no voters:
I20260811 12:02:30.643172 11100 leader_election.cc:288] T 00000000000000000000000000000000 P c2fc2c133c7c4046990cda952548edf7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:02:30.643306 11105 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c2fc2c133c7c4046990cda952548edf7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:02:30.643970 11105 raft_consensus.cc:686] T 00000000000000000000000000000000 P c2fc2c133c7c4046990cda952548edf7 [term 1 LEADER]: Becoming Leader. State: Replica: c2fc2c133c7c4046990cda952548edf7, State: Running, Role: LEADER
I20260811 12:02:30.644537 11105 consensus_queue.cc:237] T 00000000000000000000000000000000 P c2fc2c133c7c4046990cda952548edf7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2fc2c133c7c4046990cda952548edf7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 44099 } }
I20260811 12:02:30.645190 11100 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2fc2c133c7c4046990cda952548edf7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:02:30.647400 11108 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2fc2c133c7c4046990cda952548edf7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c2fc2c133c7c4046990cda952548edf7. Latest consensus state: current_term: 1 leader_uuid: "c2fc2c133c7c4046990cda952548edf7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2fc2c133c7c4046990cda952548edf7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 44099 } } }
I20260811 12:02:30.647425 11106 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2fc2c133c7c4046990cda952548edf7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c2fc2c133c7c4046990cda952548edf7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2fc2c133c7c4046990cda952548edf7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 44099 } } }
I20260811 12:02:30.648044 11106 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2fc2c133c7c4046990cda952548edf7 [sys.catalog]: This master's current role is: LEADER
I20260811 12:02:30.648810 11108 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2fc2c133c7c4046990cda952548edf7 [sys.catalog]: This master's current role is: LEADER
I20260811 12:02:30.650242 11115 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:02:30.655280 11115 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:02:30.663374 11115 catalog_manager.cc:1347] Generated new cluster ID: af33c51561a14bf389c0145173ab5650
I20260811 12:02:30.663470 11115 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:02:30.673249 11115 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:02:30.674960 11115 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:02:30.688843 11115 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c2fc2c133c7c4046990cda952548edf7: Generated new TSK 0
I20260811 12:02:30.689782 11115 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.221.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab.
Entry for principal kudu/127.9.221.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab.
I20260811 12:02:30.737648 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.129:0
--local_ip_for_outbound_sockets=127.9.221.129
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:44099
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab
--principal=kudu/127.9.221.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:42927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:02:31.062440 11138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:02:31.062736 11138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:02:31.062906 11138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:02:31.069991 11138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:02:31.070132 11138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:02:31.070341 11138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.129
I20260811 12:02:31.079605 11138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:42927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab
--principal=kudu/127.9.221.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:44099
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.129
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:02:31.080768 11138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:02:31.081947 11138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:02:31.095381 11150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:31.095357 11155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:31.095460 11152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:02:31.096158 11138 server_base.cc:1034] running on GCE node
I20260811 12:02:31.096792 11138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:02:31.097735 11138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:02:31.098935 11138 hybrid_clock.cc:648] HybridClock initialized: now 1786449751098881 us; error 44 us; skew 500 ppm
I20260811 12:02:31.099241 11138 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:02:31 dist-test-slave-mjjr krb5kdc[10968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449751, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:02:31.102506 11138 init.cc:377] Logged in from keytab as kudu/127.9.221.129@KRBTEST.COM (short username kudu)
I20260811 12:02:31.103960 11138 webserver.cc:460] Webserver started at http://127.9.221.129:43945/ using document root <none> and password file <none>
I20260811 12:02:31.104629 11138 fs_manager.cc:359] Metadata directory not provided
I20260811 12:02:31.104777 11138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:02:31.105104 11138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:02:31.108585 11138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-0/data/instance:
uuid: "dc6dd1c2bf104154a90b72dc34bbbbdf"
format_stamp: "Formatted at 2026-08-11 12:02:31 on dist-test-slave-mjjr"
server_key: "28733f26acb3a419e9a5b2e38ec4cf4c"
server_key_iv: "d8fba5f06b264a7f7cba68163be2fc9d"
server_key_version: "encryptionkey@0"
I20260811 12:02:31.109444 11138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance:
uuid: "dc6dd1c2bf104154a90b72dc34bbbbdf"
format_stamp: "Formatted at 2026-08-11 12:02:31 on dist-test-slave-mjjr"
server_key: "28733f26acb3a419e9a5b2e38ec4cf4c"
server_key_iv: "d8fba5f06b264a7f7cba68163be2fc9d"
server_key_version: "encryptionkey@0"
I20260811 12:02:31.114619 11138 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:02:31.118067 11167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:31.119287 11138 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 12:02:31.119469 11138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-0/wal
uuid: "dc6dd1c2bf104154a90b72dc34bbbbdf"
format_stamp: "Formatted at 2026-08-11 12:02:31 on dist-test-slave-mjjr"
server_key: "28733f26acb3a419e9a5b2e38ec4cf4c"
server_key_iv: "d8fba5f06b264a7f7cba68163be2fc9d"
server_key_version: "encryptionkey@0"
I20260811 12:02:31.119700 11138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:02:31.143177 11138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:02:31.147382 11138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:02:31.147727 11138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:02:31.148739 11138 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:02:31.150262 11138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:02:31.150365 11138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:31.150516 11138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:02:31.150588 11138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:31.183285 11138 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.129:33595
I20260811 12:02:31.183334 11370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.129:33595 every 8 connection(s)
I20260811 12:02:31.184588 11138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
I20260811 12:02:31.185719 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 11138
I20260811 12:02:31.185950 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance
I20260811 12:02:31.186656 10102 external_mini_cluster.cc:1428] Setting key 0259150c86998e33c38f98c9a4eee566
Aug 11 12:02:31 dist-test-slave-mjjr krb5kdc[10968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449751, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.221.130@KRBTEST.COM; defaulting to no policy
I20260811 12:02:31.206710 11376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:31.187377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:47365 (local address 127.9.221.190:44099)
0811 12:02:31.187867 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:31.187872 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:31.189300 (+ 1428us) server_negotiation.cc:407] Connection header received
0811 12:02:31.191106 (+ 1806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:31.191112 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:31.191205 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:31.191353 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:31.192917 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:31.194067 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:31.195542 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:31.196355 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:31.196498 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:31.199743 (+ 3245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:31.199753 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:31.199760 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:31.199804 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:31.203470 (+ 3666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:31.204255 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:31.204262 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:31.204267 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:31.204425 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:31.204877 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:31.204883 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:31.204886 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:31.205275 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:31.205445 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:31.206143 (+ 698us) server_negotiation.cc:299] Negotiation successful
0811 12:02:31.206400 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":232,"threads_started":1}
I20260811 12:02:31.207614 11378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:31.187934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44099 (local address 127.9.221.129:47365)
0811 12:02:31.189154 (+ 1220us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:31.189196 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:31.190798 (+ 1602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:31.191608 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:31.191615 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:31.192188 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:02:31.192747 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:31.192763 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:31.194205 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:31.194208 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:31.195379 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:31.195390 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:31.196557 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:31.196561 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:31.196755 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:31.197421 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:02:31.197449 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:02:31.199506 (+ 2057us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:02:31.203610 (+ 4104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:31.203616 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:31.203626 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:31.204131 (+ 505us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:31.204546 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:31.204549 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:31.204551 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:31.204746 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:31.205402 (+ 656us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:02:31.205407 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:02:31.205705 (+ 298us) client_negotiation.cc:769] Sending connection context
0811 12:02:31.205929 (+ 224us) client_negotiation.cc:240] Negotiation successful
0811 12:02:31.206403 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":328,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.130@KRBTEST.COM" created.
I20260811 12:02:31.209347 11373 heartbeater.cc:347] Connected to a master server at 127.9.221.190:44099
I20260811 12:02:31.209694 11373 heartbeater.cc:464] Registering TS with master...
I20260811 12:02:31.210425 11373 heartbeater.cc:511] Master 127.9.221.190:44099 requested a full tablet report, sending...
I20260811 12:02:31.212560 11039 ts_manager.cc:194] Registered new tserver with Master: dc6dd1c2bf104154a90b72dc34bbbbdf (127.9.221.129:33595)
I20260811 12:02:31.214212 11039 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.129@KRBTEST.COM'} at 127.9.221.129:47365
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab.
Entry for principal kudu/127.9.221.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab.
I20260811 12:02:31.230484 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.130:0
--local_ip_for_outbound_sockets=127.9.221.130
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:44099
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab
--principal=kudu/127.9.221.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:42927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:02:31.547880 11383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:02:31.548177 11383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:02:31.548350 11383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:02:31.555413 11383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:02:31.555554 11383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:02:31.555744 11383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.130
I20260811 12:02:31.565310 11383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:42927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab
--principal=kudu/127.9.221.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:44099
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.130
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:02:31.566565 11383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:02:31.567992 11383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:02:31.581037 11394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:31.581204 11389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:31.581080 11390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:02:31.585057 11383 server_base.cc:1034] running on GCE node
I20260811 12:02:31.585662 11383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:02:31.586587 11383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:02:31.587792 11383 hybrid_clock.cc:648] HybridClock initialized: now 1786449751587748 us; error 62 us; skew 500 ppm
I20260811 12:02:31.588106 11383 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:02:31 dist-test-slave-mjjr krb5kdc[10968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449751, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:02:31.591148 11383 init.cc:377] Logged in from keytab as kudu/127.9.221.130@KRBTEST.COM (short username kudu)
I20260811 12:02:31.592715 11383 webserver.cc:460] Webserver started at http://127.9.221.130:42507/ using document root <none> and password file <none>
I20260811 12:02:31.593410 11383 fs_manager.cc:359] Metadata directory not provided
I20260811 12:02:31.593566 11383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:02:31.593883 11383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:02:31.596920 11383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-1/data/instance:
uuid: "11f4c34f0c5c450c867db5a8a176f266"
format_stamp: "Formatted at 2026-08-11 12:02:31 on dist-test-slave-mjjr"
server_key: "b41b10d9bcfee6099d0429483d1b3fcc"
server_key_iv: "419403dc1f33e126dad0b7fb9bea90f4"
server_key_version: "encryptionkey@0"
I20260811 12:02:31.597699 11383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance:
uuid: "11f4c34f0c5c450c867db5a8a176f266"
format_stamp: "Formatted at 2026-08-11 12:02:31 on dist-test-slave-mjjr"
server_key: "b41b10d9bcfee6099d0429483d1b3fcc"
server_key_iv: "419403dc1f33e126dad0b7fb9bea90f4"
server_key_version: "encryptionkey@0"
I20260811 12:02:31.603031 11383 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260811 12:02:31.607218 11404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:31.608770 11383 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:02:31.608916 11383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-1/wal
uuid: "11f4c34f0c5c450c867db5a8a176f266"
format_stamp: "Formatted at 2026-08-11 12:02:31 on dist-test-slave-mjjr"
server_key: "b41b10d9bcfee6099d0429483d1b3fcc"
server_key_iv: "419403dc1f33e126dad0b7fb9bea90f4"
server_key_version: "encryptionkey@0"
I20260811 12:02:31.609124 11383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:02:31.634181 11383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:02:31.638021 11383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:02:31.638317 11383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:02:31.639189 11383 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:02:31.640493 11383 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:02:31.640563 11383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:31.640642 11383 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:02:31.640686 11383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:31.673590 11383 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.130:43435
I20260811 12:02:31.673676 11620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.130:43435 every 8 connection(s)
I20260811 12:02:31.674834 11383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
I20260811 12:02:31.676326 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 11383
I20260811 12:02:31.676527 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance
I20260811 12:02:31.677129 10102 external_mini_cluster.cc:1428] Setting key 9e313af396d4cc23b72e0362173115e6
Aug 11 12:02:31 dist-test-slave-mjjr krb5kdc[10968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449751, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.130@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:02:31.695874 11376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:31.677448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:34459 (local address 127.9.221.190:44099)
0811 12:02:31.677641 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:31.677645 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:31.678890 (+ 1245us) server_negotiation.cc:407] Connection header received
0811 12:02:31.680727 (+ 1837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:31.680736 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:31.680827 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:31.680938 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:31.682262 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:31.683391 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:31.685468 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:31.685862 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:31.686033 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:31.690018 (+ 3985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:31.690023 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:31.690026 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:31.690057 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:31.692602 (+ 2545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:31.693591 (+ 989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:31.693595 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:31.693597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:31.693724 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:31.694207 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:31.694210 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:31.694212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:31.694449 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:31.694555 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:31.695397 (+ 842us) server_negotiation.cc:299] Negotiation successful
0811 12:02:31.695569 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
WARNING: no policy specified for kudu/127.9.221.131@KRBTEST.COM; defaulting to no policy
I20260811 12:02:31.697115 11625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:31.677935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44099 (local address 127.9.221.130:34459)
0811 12:02:31.678718 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:31.678771 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:31.680465 (+ 1694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:31.681109 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:31.681116 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:31.681575 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:02:31.682090 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:31.682106 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:31.683534 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:31.683537 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:31.685304 (+ 1767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:31.685321 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:31.686131 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:31.686138 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:31.686375 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:31.687078 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:02:31.687099 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:02:31.689804 (+ 2705us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:02:31.692742 (+ 2938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:31.692752 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:31.692768 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:31.693469 (+ 701us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:31.693836 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:31.693841 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:31.693845 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:31.694097 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:31.694588 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:02:31.694596 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:02:31.695014 (+ 418us) client_negotiation.cc:769] Sending connection context
0811 12:02:31.695322 (+ 308us) client_negotiation.cc:240] Negotiation successful
0811 12:02:31.695898 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":334,"threads_started":1}
I20260811 12:02:31.698493 11621 heartbeater.cc:347] Connected to a master server at 127.9.221.190:44099
I20260811 12:02:31.698801 11621 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.131@KRBTEST.COM" created.
I20260811 12:02:31.699530 11621 heartbeater.cc:511] Master 127.9.221.190:44099 requested a full tablet report, sending...
I20260811 12:02:31.701012 11039 ts_manager.cc:194] Registered new tserver with Master: 11f4c34f0c5c450c867db5a8a176f266 (127.9.221.130:43435)
I20260811 12:02:31.702013 11039 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.130@KRBTEST.COM'} at 127.9.221.130:34459
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab.
Entry for principal kudu/127.9.221.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab.
I20260811 12:02:31.720898 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.131:0
--local_ip_for_outbound_sockets=127.9.221.131
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:44099
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab
--principal=kudu/127.9.221.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:42927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:02:32.071707 11630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:02:32.072037 11630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:02:32.072213 11630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:02:32.079253 11630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:02:32.079392 11630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:02:32.079579 11630 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.131
I20260811 12:02:32.089051 11630 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:42927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab
--principal=kudu/127.9.221.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:44099
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.131
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:02:32.090344 11630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:02:32.091722 11630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:02:32.104550 11640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:32.104600 11639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:32.104995 11647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:02:32.105494 11630 server_base.cc:1034] running on GCE node
I20260811 12:02:32.106145 11630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:02:32.107147 11630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:02:32.108376 11630 hybrid_clock.cc:648] HybridClock initialized: now 1786449752108342 us; error 58 us; skew 500 ppm
I20260811 12:02:32.108716 11630 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:02:32 dist-test-slave-mjjr krb5kdc[10968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449752, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:02:32.112058 11630 init.cc:377] Logged in from keytab as kudu/127.9.221.131@KRBTEST.COM (short username kudu)
I20260811 12:02:32.113446 11630 webserver.cc:460] Webserver started at http://127.9.221.131:36437/ using document root <none> and password file <none>
I20260811 12:02:32.114223 11630 fs_manager.cc:359] Metadata directory not provided
I20260811 12:02:32.114373 11630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:02:32.114710 11630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:02:32.117630 11630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-2/data/instance:
uuid: "e7b38b92c6274579a7c5cf51f98be10d"
format_stamp: "Formatted at 2026-08-11 12:02:32 on dist-test-slave-mjjr"
server_key: "5fc41f617ae16c6c937af9ca5d1672b3"
server_key_iv: "f226d95c5fb4445e0e148e039e6025fc"
server_key_version: "encryptionkey@0"
I20260811 12:02:32.118413 11630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance:
uuid: "e7b38b92c6274579a7c5cf51f98be10d"
format_stamp: "Formatted at 2026-08-11 12:02:32 on dist-test-slave-mjjr"
server_key: "5fc41f617ae16c6c937af9ca5d1672b3"
server_key_iv: "f226d95c5fb4445e0e148e039e6025fc"
server_key_version: "encryptionkey@0"
I20260811 12:02:32.123736 11630 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 12:02:32.127156 11659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:32.128648 11630 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:02:32.128821 11630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-2/wal
uuid: "e7b38b92c6274579a7c5cf51f98be10d"
format_stamp: "Formatted at 2026-08-11 12:02:32 on dist-test-slave-mjjr"
server_key: "5fc41f617ae16c6c937af9ca5d1672b3"
server_key_iv: "f226d95c5fb4445e0e148e039e6025fc"
server_key_version: "encryptionkey@0"
I20260811 12:02:32.129017 11630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:02:32.145437 11630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:02:32.149891 11630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:02:32.150198 11630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:02:32.151211 11630 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:02:32.152647 11630 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:02:32.152720 11630 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:32.152873 11630 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:02:32.152925 11630 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:32.187011 11630 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.131:44055
I20260811 12:02:32.187021 11853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.131:44055 every 8 connection(s)
I20260811 12:02:32.188299 11630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
I20260811 12:02:32.190088 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 11630
I20260811 12:02:32.190285 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance
I20260811 12:02:32.190743 10102 external_mini_cluster.cc:1428] Setting key 75ee354b50cb4646b950d3e0773c5899
Aug 11 12:02:32 dist-test-slave-mjjr krb5kdc[10968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449752, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.131@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:02:32.207908 11376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:32.191014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:56489 (local address 127.9.221.190:44099)
0811 12:02:32.191232 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:32.191238 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:32.192498 (+ 1260us) server_negotiation.cc:407] Connection header received
0811 12:02:32.194584 (+ 2086us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:32.194588 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:32.194651 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:32.194732 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:32.196158 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:32.196924 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:32.198417 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:32.198834 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:32.198931 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:32.202172 (+ 3241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:32.202177 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:32.202180 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:32.202211 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:32.204907 (+ 2696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:32.205627 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:32.205631 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:32.205633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:32.205749 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:32.206194 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:32.206199 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:32.206204 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:32.206495 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:32.206655 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:32.207314 (+ 659us) server_negotiation.cc:299] Negotiation successful
0811 12:02:32.207534 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 12:02:32.208789 11857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:32.191403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44099 (local address 127.9.221.131:56489)
0811 12:02:32.192320 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:32.192368 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:32.194350 (+ 1982us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:32.194937 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:32.194944 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:32.195400 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:02:32.196020 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:32.196037 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:32.197066 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:32.197069 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:32.198297 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:32.198308 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:32.199013 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:32.199018 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:32.199223 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:32.199896 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:02:32.199916 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:02:32.201983 (+ 2067us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:02:32.205050 (+ 3067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:32.205058 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:32.205073 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:32.205487 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:32.205863 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:32.205868 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:32.205871 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:32.206070 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:32.206630 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:02:32.206638 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:02:32.206958 (+ 320us) client_negotiation.cc:769] Sending connection context
0811 12:02:32.207213 (+ 255us) client_negotiation.cc:240] Negotiation successful
0811 12:02:32.207753 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":355,"threads_started":1}
I20260811 12:02:32.210171 11854 heartbeater.cc:347] Connected to a master server at 127.9.221.190:44099
I20260811 12:02:32.210485 11854 heartbeater.cc:464] Registering TS with master...
I20260811 12:02:32.211244 11854 heartbeater.cc:511] Master 127.9.221.190:44099 requested a full tablet report, sending...
I20260811 12:02:32.212692 11039 ts_manager.cc:194] Registered new tserver with Master: e7b38b92c6274579a7c5cf51f98be10d (127.9.221.131:44055)
I20260811 12:02:32.213524 11039 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.131@KRBTEST.COM'} at 127.9.221.131:56489
I20260811 12:02:32.217154 11373 heartbeater.cc:503] Master 127.9.221.190:44099 was elected leader, sending a full tablet report...
I20260811 12:02:32.218614 10102 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:02:32.238178 11376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:32.225560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56584 (local address 127.9.221.190:44099)
0811 12:02:32.225741 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:32.225750 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:32.226178 (+ 428us) server_negotiation.cc:407] Connection header received
0811 12:02:32.226371 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:32.226376 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:32.226450 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:32.226564 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:32.228132 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:32.228826 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:32.230061 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:32.230491 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:32.230628 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:32.233314 (+ 2686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:32.233319 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:32.233322 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:32.233395 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:32.235940 (+ 2545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:32.236585 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:32.236588 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:32.236591 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:32.236710 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:32.237057 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:32.237060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:32.237062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:32.237353 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:32.237525 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:32.237797 (+ 272us) server_negotiation.cc:299] Negotiation successful
0811 12:02:32.237949 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 12:02:32.246161 11039 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56584:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:02:32.254057 11039 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:02:32.283136 11878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:32.269927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45612 (local address 127.9.221.129:33595)
0811 12:02:32.271266 (+ 1339us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:32.271275 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:32.271305 (+ 30us) server_negotiation.cc:407] Connection header received
0811 12:02:32.271407 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:32.271414 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:32.271624 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:32.272626 (+ 1002us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:02:32.272892 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:32.274872 (+ 1980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:32.279147 (+ 4275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:32.281115 (+ 1968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:32.282308 (+ 1193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:32.282549 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:32.282630 (+ 81us) server_negotiation.cc:299] Negotiation successful
0811 12:02:32.282797 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":835,"spinlock_wait_cycles":552832,"thread_start_us":865,"threads_started":1}
I20260811 12:02:32.283352 11877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:32.269557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:33595 (local address 127.0.0.1:45612)
0811 12:02:32.270205 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:32.270224 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:32.270329 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:32.271887 (+ 1558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:32.271894 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:32.271923 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:02:32.272362 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:32.272379 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:32.275068 (+ 2689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:32.275074 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:32.278825 (+ 3751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:32.278850 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:32.281491 (+ 2641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:32.281501 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:32.281865 (+ 364us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:32.282007 (+ 142us) client_negotiation.cc:769] Sending connection context
0811 12:02:32.282221 (+ 214us) client_negotiation.cc:240] Negotiation successful
0811 12:02:32.282539 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"mutex_wait_us":110,"spinlock_wait_cycles":896,"thread_start_us":364,"threads_started":1}
I20260811 12:02:32.285693 11875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:32.268998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58994 (local address 127.9.221.130:43435)
0811 12:02:32.269539 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:32.269550 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:32.270172 (+ 622us) server_negotiation.cc:407] Connection header received
0811 12:02:32.270620 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:32.270631 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:32.270875 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:32.271053 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:02:32.273195 (+ 2142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:32.277217 (+ 4022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:32.282735 (+ 5518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:32.284298 (+ 1563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:32.284475 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:32.284754 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:32.285168 (+ 414us) server_negotiation.cc:299] Negotiation successful
0811 12:02:32.285383 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":289,"threads_started":1}
I20260811 12:02:32.286825 11876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:32.268938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.130:43435 (local address 127.0.0.1:58994)
0811 12:02:32.269993 (+ 1055us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:32.270051 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:32.270329 (+ 278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:32.272184 (+ 1855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:32.272189 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:32.272218 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:02:32.272839 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:32.272855 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:32.277429 (+ 4574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:32.277434 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:32.282552 (+ 5118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:32.282567 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:32.284601 (+ 2034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:32.284609 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:32.284862 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:32.284905 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 12:02:32.285881 (+ 976us) client_negotiation.cc:240] Negotiation successful
0811 12:02:32.286066 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":776,"mutex_wait_us":107,"thread_start_us":1150,"threads_started":1}
I20260811 12:02:32.288836 11247 tablet_service.cc:1467] Processing CreateTablet for tablet 31fe671384a148b4a85fcb25cea45aeb (DEFAULT_TABLE table=test-table [id=f663422607164ba0b09f23b3753bd2f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:02:32.290975 11247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31fe671384a148b4a85fcb25cea45aeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:02:32.291443 11487 tablet_service.cc:1467] Processing CreateTablet for tablet 31fe671384a148b4a85fcb25cea45aeb (DEFAULT_TABLE table=test-table [id=f663422607164ba0b09f23b3753bd2f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:02:32.292696 11883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:32.270672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.131:44055 (local address 127.0.0.1:33420)
0811 12:02:32.281285 (+ 10613us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:32.281306 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:32.281471 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:32.283808 (+ 2337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:32.283812 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:32.283838 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:02:32.284212 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:32.284237 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:32.286899 (+ 2662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:32.286908 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:32.289335 (+ 2427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:32.289348 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:32.291555 (+ 2207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:32.291562 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:32.292052 (+ 490us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:32.292099 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 12:02:32.292196 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 12:02:32.292329 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10359,"thread_start_us":10067,"threads_started":1}
I20260811 12:02:32.293186 11487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31fe671384a148b4a85fcb25cea45aeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:02:32.293622 11884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:32.271027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33420 (local address 127.9.221.131:44055)
0811 12:02:32.282521 (+ 11494us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:32.282529 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:32.282580 (+ 51us) server_negotiation.cc:407] Connection header received
0811 12:02:32.282726 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:32.282735 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:32.283020 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:32.283337 (+ 317us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:02:32.284629 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:32.286719 (+ 2090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:32.289531 (+ 2812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:32.290750 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:32.290897 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:32.291177 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:32.293047 (+ 1870us) server_negotiation.cc:299] Negotiation successful
0811 12:02:32.293158 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11278,"thread_start_us":11002,"threads_started":1}
I20260811 12:02:32.296978 11748 tablet_service.cc:1467] Processing CreateTablet for tablet 31fe671384a148b4a85fcb25cea45aeb (DEFAULT_TABLE table=test-table [id=f663422607164ba0b09f23b3753bd2f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:02:32.299616 11748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31fe671384a148b4a85fcb25cea45aeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:02:32.312966 11885 tablet_bootstrap.cc:492] T 31fe671384a148b4a85fcb25cea45aeb P dc6dd1c2bf104154a90b72dc34bbbbdf: Bootstrap starting.
I20260811 12:02:32.317158 11889 tablet_bootstrap.cc:492] T 31fe671384a148b4a85fcb25cea45aeb P e7b38b92c6274579a7c5cf51f98be10d: Bootstrap starting.
I20260811 12:02:32.318503 11886 tablet_bootstrap.cc:492] T 31fe671384a148b4a85fcb25cea45aeb P 11f4c34f0c5c450c867db5a8a176f266: Bootstrap starting.
I20260811 12:02:32.322227 11885 tablet_bootstrap.cc:654] T 31fe671384a148b4a85fcb25cea45aeb P dc6dd1c2bf104154a90b72dc34bbbbdf: Neither blocks nor log segments found. Creating new log.
I20260811 12:02:32.322395 11889 tablet_bootstrap.cc:654] T 31fe671384a148b4a85fcb25cea45aeb P e7b38b92c6274579a7c5cf51f98be10d: Neither blocks nor log segments found. Creating new log.
I20260811 12:02:32.324384 11886 tablet_bootstrap.cc:654] T 31fe671384a148b4a85fcb25cea45aeb P 11f4c34f0c5c450c867db5a8a176f266: Neither blocks nor log segments found. Creating new log.
I20260811 12:02:32.324440 11889 log.cc:824] T 31fe671384a148b4a85fcb25cea45aeb P e7b38b92c6274579a7c5cf51f98be10d: Log is configured to *not* fsync() on all Append() calls
I20260811 12:02:32.324623 11885 log.cc:824] T 31fe671384a148b4a85fcb25cea45aeb P dc6dd1c2bf104154a90b72dc34bbbbdf: Log is configured to *not* fsync() on all Append() calls
I20260811 12:02:32.326670 11886 log.cc:824] T 31fe671384a148b4a85fcb25cea45aeb P 11f4c34f0c5c450c867db5a8a176f266: Log is configured to *not* fsync() on all Append() calls
I20260811 12:02:32.329416 11885 tablet_bootstrap.cc:492] T 31fe671384a148b4a85fcb25cea45aeb P dc6dd1c2bf104154a90b72dc34bbbbdf: No bootstrap required, opened a new log
I20260811 12:02:32.329630 11889 tablet_bootstrap.cc:492] T 31fe671384a148b4a85fcb25cea45aeb P e7b38b92c6274579a7c5cf51f98be10d: No bootstrap required, opened a new log
I20260811 12:02:32.329819 11885 ts_tablet_manager.cc:1397] T 31fe671384a148b4a85fcb25cea45aeb P dc6dd1c2bf104154a90b72dc34bbbbdf: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.006s
I20260811 12:02:32.330050 11889 ts_tablet_manager.cc:1397] T 31fe671384a148b4a85fcb25cea45aeb P e7b38b92c6274579a7c5cf51f98be10d: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.009s
I20260811 12:02:32.331194 11886 tablet_bootstrap.cc:492] T 31fe671384a148b4a85fcb25cea45aeb P 11f4c34f0c5c450c867db5a8a176f266: No bootstrap required, opened a new log
I20260811 12:02:32.331606 11886 ts_tablet_manager.cc:1397] T 31fe671384a148b4a85fcb25cea45aeb P 11f4c34f0c5c450c867db5a8a176f266: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.004s
I20260811 12:02:32.341572 11885 raft_consensus.cc:348] T 31fe671384a148b4a85fcb25cea45aeb P dc6dd1c2bf104154a90b72dc34bbbbdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f4c34f0c5c450c867db5a8a176f266" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43435 } } peers { permanent_uuid: "e7b38b92c6274579a7c5cf51f98be10d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 44055 } } peers { permanent_uuid: "dc6dd1c2bf104154a90b72dc34bbbbdf" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33595 } }
I20260811 12:02:32.342134 11885 raft_consensus.cc:374] T 31fe671384a148b4a85fcb25cea45aeb P dc6dd1c2bf104154a90b72dc34bbbbdf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:02:32.342336 11885 raft_consensus.cc:729] T 31fe671384a148b4a85fcb25cea45aeb P dc6dd1c2bf104154a90b72dc34bbbbdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc6dd1c2bf104154a90b72dc34bbbbdf, State: Initialized, Role: FOLLOWER
I20260811 12:02:32.343103 11885 consensus_queue.cc:260] T 31fe671384a148b4a85fcb25cea45aeb P dc6dd1c2bf104154a90b72dc34bbbbdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f4c34f0c5c450c867db5a8a176f266" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43435 } } peers { permanent_uuid: "e7b38b92c6274579a7c5cf51f98be10d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 44055 } } peers { permanent_uuid: "dc6dd1c2bf104154a90b72dc34bbbbdf" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33595 } }
I20260811 12:02:32.345680 11885 ts_tablet_manager.cc:1428] T 31fe671384a148b4a85fcb25cea45aeb P dc6dd1c2bf104154a90b72dc34bbbbdf: Time spent starting tablet: real 0.016s user 0.016s sys 0.000s
I20260811 12:02:32.347328 11889 raft_consensus.cc:348] T 31fe671384a148b4a85fcb25cea45aeb P e7b38b92c6274579a7c5cf51f98be10d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f4c34f0c5c450c867db5a8a176f266" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43435 } } peers { permanent_uuid: "e7b38b92c6274579a7c5cf51f98be10d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 44055 } } peers { permanent_uuid: "dc6dd1c2bf104154a90b72dc34bbbbdf" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33595 } }
I20260811 12:02:32.348034 11889 raft_consensus.cc:374] T 31fe671384a148b4a85fcb25cea45aeb P e7b38b92c6274579a7c5cf51f98be10d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:02:32.348292 11889 raft_consensus.cc:729] T 31fe671384a148b4a85fcb25cea45aeb P e7b38b92c6274579a7c5cf51f98be10d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7b38b92c6274579a7c5cf51f98be10d, State: Initialized, Role: FOLLOWER
I20260811 12:02:32.348294 11886 raft_consensus.cc:348] T 31fe671384a148b4a85fcb25cea45aeb P 11f4c34f0c5c450c867db5a8a176f266 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f4c34f0c5c450c867db5a8a176f266" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43435 } } peers { permanent_uuid: "e7b38b92c6274579a7c5cf51f98be10d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 44055 } } peers { permanent_uuid: "dc6dd1c2bf104154a90b72dc34bbbbdf" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33595 } }
I20260811 12:02:32.348987 11886 raft_consensus.cc:374] T 31fe671384a148b4a85fcb25cea45aeb P 11f4c34f0c5c450c867db5a8a176f266 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:02:32.349254 11886 raft_consensus.cc:729] T 31fe671384a148b4a85fcb25cea45aeb P 11f4c34f0c5c450c867db5a8a176f266 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f4c34f0c5c450c867db5a8a176f266, State: Initialized, Role: FOLLOWER
I20260811 12:02:32.349125 11889 consensus_queue.cc:260] T 31fe671384a148b4a85fcb25cea45aeb P e7b38b92c6274579a7c5cf51f98be10d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f4c34f0c5c450c867db5a8a176f266" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43435 } } peers { permanent_uuid: "e7b38b92c6274579a7c5cf51f98be10d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 44055 } } peers { permanent_uuid: "dc6dd1c2bf104154a90b72dc34bbbbdf" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33595 } }
I20260811 12:02:32.350144 11886 consensus_queue.cc:260] T 31fe671384a148b4a85fcb25cea45aeb P 11f4c34f0c5c450c867db5a8a176f266 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f4c34f0c5c450c867db5a8a176f266" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43435 } } peers { permanent_uuid: "e7b38b92c6274579a7c5cf51f98be10d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 44055 } } peers { permanent_uuid: "dc6dd1c2bf104154a90b72dc34bbbbdf" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33595 } }
I20260811 12:02:32.351585 11854 heartbeater.cc:503] Master 127.9.221.190:44099 was elected leader, sending a full tablet report...
I20260811 12:02:32.352463 11889 ts_tablet_manager.cc:1428] T 31fe671384a148b4a85fcb25cea45aeb P e7b38b92c6274579a7c5cf51f98be10d: Time spent starting tablet: real 0.022s user 0.024s sys 0.000s
I20260811 12:02:32.353986 11621 heartbeater.cc:503] Master 127.9.221.190:44099 was elected leader, sending a full tablet report...
I20260811 12:02:32.354856 11886 ts_tablet_manager.cc:1428] T 31fe671384a148b4a85fcb25cea45aeb P 11f4c34f0c5c450c867db5a8a176f266: Time spent starting tablet: real 0.023s user 0.018s sys 0.004s
I20260811 12:02:32.417613 11901 raft_consensus.cc:482] T 31fe671384a148b4a85fcb25cea45aeb P 11f4c34f0c5c450c867db5a8a176f266 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:02:32.418154 11901 raft_consensus.cc:504] T 31fe671384a148b4a85fcb25cea45aeb P 11f4c34f0c5c450c867db5a8a176f266 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f4c34f0c5c450c867db5a8a176f266" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43435 } } peers { permanent_uuid: "e7b38b92c6274579a7c5cf51f98be10d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 44055 } } peers { permanent_uuid: "dc6dd1c2bf104154a90b72dc34bbbbdf" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33595 } }
I20260811 12:02:32.420989 11901 leader_election.cc:288] T 31fe671384a148b4a85fcb25cea45aeb P 11f4c34f0c5c450c867db5a8a176f266 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7b38b92c6274579a7c5cf51f98be10d (127.9.221.131:44055), dc6dd1c2bf104154a90b72dc34bbbbdf (127.9.221.129:33595)
W20260811 12:02:32.426887 11623 tablet.cc:2369] T 31fe671384a148b4a85fcb25cea45aeb P 11f4c34f0c5c450c867db5a8a176f266: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:02:32.432742 11878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:32.421856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:42731 (local address 127.9.221.129:33595)
0811 12:02:32.422267 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:32.422274 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:32.423954 (+ 1680us) server_negotiation.cc:407] Connection header received
0811 12:02:32.424171 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:32.424178 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:32.424255 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:32.424352 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:02:32.425900 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:32.427379 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:32.430271 (+ 2892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:32.431457 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:32.431687 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:32.431811 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:32.432240 (+ 429us) server_negotiation.cc:299] Negotiation successful
0811 12:02:32.432402 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160}
I20260811 12:02:32.432857 11884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:32.421593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:49653 (local address 127.9.221.131:44055)
0811 12:02:32.421950 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:32.421957 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:32.423726 (+ 1769us) server_negotiation.cc:407] Connection header received
0811 12:02:32.424037 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:32.424041 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:32.424132 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:32.424233 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:02:32.425699 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:32.427383 (+ 1684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:32.430427 (+ 3044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:32.431480 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:32.431734 (+ 254us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:32.431854 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:32.432417 (+ 563us) server_negotiation.cc:299] Negotiation successful
0811 12:02:32.432577 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176}
I20260811 12:02:32.433347 11903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:32.421931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:33595 (local address 127.9.221.130:42731)
0811 12:02:32.423863 (+ 1932us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:32.423880 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:32.424022 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:32.424380 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:32.424394 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:32.424428 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:02:32.425613 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:32.425626 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:32.427558 (+ 1932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:32.427567 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:32.430073 (+ 2506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:32.430089 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:32.431847 (+ 1758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:32.431854 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:32.432082 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:32.432108 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 12:02:32.432189 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 12:02:32.432274 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1753,"mutex_wait_us":5,"thread_start_us":1352,"threads_started":1}
I20260811 12:02:32.433420 11902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:32.421105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.131:44055 (local address 127.9.221.130:49653)
0811 12:02:32.423580 (+ 2475us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:32.423620 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:32.423863 (+ 243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:32.424257 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:32.424260 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:32.424282 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:02:32.425505 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:32.425531 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:32.427563 (+ 2032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:32.427567 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:32.430234 (+ 2667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:32.430251 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:32.431999 (+ 1748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:32.432006 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:32.432211 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:32.432241 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 12:02:32.432335 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 12:02:32.432459 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2169,"mutex_wait_us":31,"thread_start_us":1626,"threads_started":1}
I20260811 12:02:32.434728 11284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31fe671384a148b4a85fcb25cea45aeb" candidate_uuid: "11f4c34f0c5c450c867db5a8a176f266" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc6dd1c2bf104154a90b72dc34bbbbdf" is_pre_election: true
I20260811 12:02:32.435129 11782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31fe671384a148b4a85fcb25cea45aeb" candidate_uuid: "11f4c34f0c5c450c867db5a8a176f266" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7b38b92c6274579a7c5cf51f98be10d" is_pre_election: true
I20260811 12:02:32.436044 11284 raft_consensus.cc:2454] T 31fe671384a148b4a85fcb25cea45aeb P dc6dd1c2bf104154a90b72dc34bbbbdf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f4c34f0c5c450c867db5a8a176f266 in term 0.
I20260811 12:02:32.436008 11782 raft_consensus.cc:2454] T 31fe671384a148b4a85fcb25cea45aeb P e7b38b92c6274579a7c5cf51f98be10d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f4c34f0c5c450c867db5a8a176f266 in term 0.
W20260811 12:02:32.437057 11374 tablet.cc:2369] T 31fe671384a148b4a85fcb25cea45aeb P dc6dd1c2bf104154a90b72dc34bbbbdf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:02:32.437112 11407 leader_election.cc:302] T 31fe671384a148b4a85fcb25cea45aeb P 11f4c34f0c5c450c867db5a8a176f266 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f4c34f0c5c450c867db5a8a176f266, dc6dd1c2bf104154a90b72dc34bbbbdf; no voters:
I20260811 12:02:32.437641 11901 raft_consensus.cc:2781] T 31fe671384a148b4a85fcb25cea45aeb P 11f4c34f0c5c450c867db5a8a176f266 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:02:32.437882 11901 raft_consensus.cc:482] T 31fe671384a148b4a85fcb25cea45aeb P 11f4c34f0c5c450c867db5a8a176f266 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:02:32.438046 11901 raft_consensus.cc:3037] T 31fe671384a148b4a85fcb25cea45aeb P 11f4c34f0c5c450c867db5a8a176f266 [term 0 FOLLOWER]: Advancing to term 1
W20260811 12:02:32.439957 11856 tablet.cc:2369] T 31fe671384a148b4a85fcb25cea45aeb P e7b38b92c6274579a7c5cf51f98be10d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:02:32.441277 11901 raft_consensus.cc:504] T 31fe671384a148b4a85fcb25cea45aeb P 11f4c34f0c5c450c867db5a8a176f266 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f4c34f0c5c450c867db5a8a176f266" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43435 } } peers { permanent_uuid: "e7b38b92c6274579a7c5cf51f98be10d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 44055 } } peers { permanent_uuid: "dc6dd1c2bf104154a90b72dc34bbbbdf" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33595 } }
I20260811 12:02:32.442075 11901 leader_election.cc:288] T 31fe671384a148b4a85fcb25cea45aeb P 11f4c34f0c5c450c867db5a8a176f266 [CANDIDATE]: Term 1 election: Requested vote from peers e7b38b92c6274579a7c5cf51f98be10d (127.9.221.131:44055), dc6dd1c2bf104154a90b72dc34bbbbdf (127.9.221.129:33595)
I20260811 12:02:32.442469 11782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31fe671384a148b4a85fcb25cea45aeb" candidate_uuid: "11f4c34f0c5c450c867db5a8a176f266" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7b38b92c6274579a7c5cf51f98be10d"
I20260811 12:02:32.442591 11284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31fe671384a148b4a85fcb25cea45aeb" candidate_uuid: "11f4c34f0c5c450c867db5a8a176f266" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc6dd1c2bf104154a90b72dc34bbbbdf"
I20260811 12:02:32.442678 11782 raft_consensus.cc:3037] T 31fe671384a148b4a85fcb25cea45aeb P e7b38b92c6274579a7c5cf51f98be10d [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:02:32.442867 11284 raft_consensus.cc:3037] T 31fe671384a148b4a85fcb25cea45aeb P dc6dd1c2bf104154a90b72dc34bbbbdf [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:02:32.445962 11782 raft_consensus.cc:2454] T 31fe671384a148b4a85fcb25cea45aeb P e7b38b92c6274579a7c5cf51f98be10d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f4c34f0c5c450c867db5a8a176f266 in term 1.
I20260811 12:02:32.446322 11284 raft_consensus.cc:2454] T 31fe671384a148b4a85fcb25cea45aeb P dc6dd1c2bf104154a90b72dc34bbbbdf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f4c34f0c5c450c867db5a8a176f266 in term 1.
I20260811 12:02:32.446480 11406 leader_election.cc:302] T 31fe671384a148b4a85fcb25cea45aeb P 11f4c34f0c5c450c867db5a8a176f266 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f4c34f0c5c450c867db5a8a176f266, e7b38b92c6274579a7c5cf51f98be10d; no voters:
I20260811 12:02:32.446830 11901 raft_consensus.cc:2781] T 31fe671384a148b4a85fcb25cea45aeb P 11f4c34f0c5c450c867db5a8a176f266 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:02:32.447603 11901 raft_consensus.cc:686] T 31fe671384a148b4a85fcb25cea45aeb P 11f4c34f0c5c450c867db5a8a176f266 [term 1 LEADER]: Becoming Leader. State: Replica: 11f4c34f0c5c450c867db5a8a176f266, State: Running, Role: LEADER
I20260811 12:02:32.448220 11901 consensus_queue.cc:237] T 31fe671384a148b4a85fcb25cea45aeb P 11f4c34f0c5c450c867db5a8a176f266 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f4c34f0c5c450c867db5a8a176f266" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43435 } } peers { permanent_uuid: "e7b38b92c6274579a7c5cf51f98be10d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 44055 } } peers { permanent_uuid: "dc6dd1c2bf104154a90b72dc34bbbbdf" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33595 } }
I20260811 12:02:32.453928 11038 catalog_manager.cc:5495] T 31fe671384a148b4a85fcb25cea45aeb P 11f4c34f0c5c450c867db5a8a176f266 reported cstate change: term changed from 0 to 1, leader changed from <none> to 11f4c34f0c5c450c867db5a8a176f266 (127.9.221.130). New cstate: current_term: 1 leader_uuid: "11f4c34f0c5c450c867db5a8a176f266" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f4c34f0c5c450c867db5a8a176f266" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43435 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e7b38b92c6274579a7c5cf51f98be10d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 44055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc6dd1c2bf104154a90b72dc34bbbbdf" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33595 } health_report { overall_health: UNKNOWN } } }
Aug 11 12:02:32 dist-test-slave-mjjr krb5kdc[10968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449750, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.129@KRBTEST.COM
I20260811 12:02:32.482535 11878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:32.462782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45624 (local address 127.9.221.129:33595)
0811 12:02:32.462940 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:32.462945 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:32.463967 (+ 1022us) server_negotiation.cc:407] Connection header received
0811 12:02:32.464131 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:32.464135 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:32.464194 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:32.464274 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:32.466600 (+ 2326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:32.467618 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:32.469171 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:32.469706 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:32.469798 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:32.474718 (+ 4920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:32.474733 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:32.474745 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:32.474775 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:32.479170 (+ 4395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:32.480009 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:32.480017 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:32.480019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:32.480206 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:32.480651 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:32.480655 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:32.480656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:32.481050 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:32.481507 (+ 457us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:32.481806 (+ 299us) server_negotiation.cc:299] Negotiation successful
0811 12:02:32.482088 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
Aug 11 12:02:32 dist-test-slave-mjjr krb5kdc[10968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449752, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 12:02:32 dist-test-slave-mjjr krb5kdc[10968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449752, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.221.129@KRBTEST.COM
I20260811 12:02:32.536129 11878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:32.519262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45626 (local address 127.9.221.129:33595)
0811 12:02:32.519492 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:32.519496 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:32.519846 (+ 350us) server_negotiation.cc:407] Connection header received
0811 12:02:32.520086 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:32.520090 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:32.520153 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:32.520236 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:32.522537 (+ 2301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:32.523345 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:32.524969 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:32.525366 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:32.525461 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:32.530116 (+ 4655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:32.530121 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:32.530124 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:32.530155 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:32.533728 (+ 3573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:32.534282 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:32.534286 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:32.534288 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:32.534439 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:32.534791 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:32.534794 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:32.534796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:32.535151 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:32.535393 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:32.535567 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 12:02:32.535824 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
Aug 11 12:02:32 dist-test-slave-mjjr krb5kdc[10968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 12:02:32 dist-test-slave-mjjr krb5kdc[10968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.129@KRBTEST.COM
I20260811 12:02:32.588919 11878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:32.570531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45630 (local address 127.9.221.129:33595)
0811 12:02:32.570787 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:32.570802 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:32.571358 (+ 556us) server_negotiation.cc:407] Connection header received
0811 12:02:32.571717 (+ 359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:32.571725 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:32.571805 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:32.571938 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:32.573916 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:32.574744 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:32.576403 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:32.577054 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:32.577198 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:32.582352 (+ 5154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:32.582357 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:32.582361 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:32.582397 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:32.585578 (+ 3181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:32.586354 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:32.586361 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:32.586367 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:32.586545 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:32.586942 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:32.586948 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:32.586951 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:32.587325 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:32.587728 (+ 403us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:32.588089 (+ 361us) server_negotiation.cc:299] Negotiation successful
0811 12:02:32.588410 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260811 12:02:32.591337 11208 tablet_service.cc:2908] Rejecting scan request for tablet 31fe671384a148b4a85fcb25cea45aeb: Uninitialized: clean time has not yet been initialized
I20260811 12:02:32.595366 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 11138
I20260811 12:02:32.611203 11878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:32.596453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45638 (local address 127.9.221.129:33595)
0811 12:02:32.596629 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:32.596633 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:32.596984 (+ 351us) server_negotiation.cc:407] Connection header received
0811 12:02:32.597133 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:32.597137 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:32.597194 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:32.597283 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:32.598363 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:32.599046 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:32.600305 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:32.600739 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:32.600897 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:32.603717 (+ 2820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:32.603724 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:32.603729 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:32.603767 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:32.608105 (+ 4338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:32.608807 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:32.608813 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:32.608817 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:32.608989 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:32.609402 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:32.609415 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:32.609419 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:32.609914 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:32.610336 (+ 422us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:32.610490 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 12:02:32.610739 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 12:02:32.612059 11363 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:02:32.710798 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 11138
I20260811 12:02:32.736138 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 11383
Aug 11 12:02:32 dist-test-slave-mjjr krb5kdc[10968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.130@KRBTEST.COM
I20260811 12:02:32.751437 11875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:32.737043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59008 (local address 127.9.221.130:43435)
0811 12:02:32.737288 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:32.737294 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:32.737315 (+ 21us) server_negotiation.cc:407] Connection header received
0811 12:02:32.737383 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:32.737386 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:32.737460 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:32.737566 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:32.738812 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:32.739531 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:32.740885 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:32.741303 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:32.741444 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:32.745823 (+ 4379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:32.745835 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:32.745849 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:32.745889 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:32.748732 (+ 2843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:32.749495 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:32.749502 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:32.749504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:32.749661 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:32.750067 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:32.750070 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:32.750072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:32.750515 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:32.750785 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:32.750985 (+ 200us) server_negotiation.cc:299] Negotiation successful
0811 12:02:32.751181 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 12:02:32.752199 11610 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:02:32.844769 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 11383
I20260811 12:02:32.870451 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 11630
Aug 11 12:02:32 dist-test-slave-mjjr krb5kdc[10968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.131@KRBTEST.COM
I20260811 12:02:32.885356 11884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:32.871244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33434 (local address 127.9.221.131:44055)
0811 12:02:32.871450 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:32.871457 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:32.871478 (+ 21us) server_negotiation.cc:407] Connection header received
0811 12:02:32.871818 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:32.871823 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:32.871887 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:32.872015 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:32.873232 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:32.874000 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:32.875314 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:32.875760 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:32.875893 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:32.880192 (+ 4299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:32.880206 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:32.880221 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:32.880251 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:32.882985 (+ 2734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:32.883572 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:32.883579 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:32.883581 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:32.883810 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:32.884129 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:32.884132 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:32.884134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:32.884473 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:32.884678 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:32.884942 (+ 264us) server_negotiation.cc:299] Negotiation successful
0811 12:02:32.885100 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 12:02:32.885967 11849 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:02:32.972492 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 11630
I20260811 12:02:32.996934 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 10986
I20260811 12:02:32.998147 11094 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:02:33.107623 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 10986
2026-08-11T12:02:33Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4418 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:02:33 dist-test-slave-mjjr krb5kdc[11933](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:02:33 dist-test-slave-mjjr krb5kdc[11933](info): set up 2 sockets
Aug 11 12:02:33 dist-test-slave-mjjr krb5kdc[11933](info): commencing operation
krb5kdc: starting...
W20260811 12:02:34.427935 10102 mini_kdc.cc:121] Time spent starting KDC: real 1.271s user 0.002s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Aug 11 12:02:34 dist-test-slave-mjjr krb5kdc[11933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:02:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:02:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.221.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
I20260811 12:02:34.613667 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:43507
--webserver_interface=127.9.221.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:34173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:43507
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:02:34.934131 11954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:02:34.934401 11954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:02:34.934518 11954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:02:34.941704 11954 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:02:34.941815 11954 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 12:02:34.941864 11954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:02:34.941949 11954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:02:34.942032 11954 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:02:34.942127 11954 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:02:34.952149 11954 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:34173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:43507
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:43507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.221.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:02:34.953444 11954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:02:34.954834 11954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:02:34.966810 11963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:34.966902 11966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:34.966857 11962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:02:34.968168 11954 server_base.cc:1034] running on GCE node
I20260811 12:02:34.968911 11954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:02:34.970305 11954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:02:34.971553 11954 hybrid_clock.cc:648] HybridClock initialized: now 1786449754971493 us; error 41 us; skew 500 ppm
I20260811 12:02:34.971894 11954 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:02:34 dist-test-slave-mjjr krb5kdc[11933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449754, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:02:34.979369 11954 init.cc:377] Logged in from keytab as kudu/127.9.221.190@KRBTEST.COM (short username kudu)
I20260811 12:02:34.980965 11954 webserver.cc:460] Webserver started at http://127.9.221.190:45563/ using document root <none> and password file <none>
I20260811 12:02:34.981667 11954 fs_manager.cc:359] Metadata directory not provided
I20260811 12:02:34.981805 11954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:02:34.982143 11954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:02:34.985193 11954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "186edf6d1f844a7c942fc1382fa1c773"
format_stamp: "Formatted at 2026-08-11 12:02:34 on dist-test-slave-mjjr"
server_key: "32b1aeaf5a8ae5add7859d92624896a9"
server_key_iv: "c5ed02a2fc2f78b32ac74c5cde1099bf"
server_key_version: "encryptionkey@0"
I20260811 12:02:34.985970 11954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "186edf6d1f844a7c942fc1382fa1c773"
format_stamp: "Formatted at 2026-08-11 12:02:34 on dist-test-slave-mjjr"
server_key: "32b1aeaf5a8ae5add7859d92624896a9"
server_key_iv: "c5ed02a2fc2f78b32ac74c5cde1099bf"
server_key_version: "encryptionkey@0"
I20260811 12:02:34.991132 11954 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:02:34.994552 11977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:34.996134 11954 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:02:34.996272 11954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "186edf6d1f844a7c942fc1382fa1c773"
format_stamp: "Formatted at 2026-08-11 12:02:34 on dist-test-slave-mjjr"
server_key: "32b1aeaf5a8ae5add7859d92624896a9"
server_key_iv: "c5ed02a2fc2f78b32ac74c5cde1099bf"
server_key_version: "encryptionkey@0"
I20260811 12:02:34.996446 11954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:02:35.033258 11954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:02:35.037496 11954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:02:35.037716 11954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:02:35.059115 11954 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.190:43507
I20260811 12:02:35.059150 12085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.190:43507 every 8 connection(s)
I20260811 12:02:35.060456 11954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:02:35.061792 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 11954
I20260811 12:02:35.062032 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:02:35.062692 10102 external_mini_cluster.cc:1428] Setting key 189b848570a0cf87fdafb7b84862bc83
I20260811 12:02:35.064785 12086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 12:02:35 dist-test-slave-mjjr krb5kdc[11933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:02:35.077979 12086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 186edf6d1f844a7c942fc1382fa1c773: Bootstrap starting.
I20260811 12:02:35.082301 12086 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 186edf6d1f844a7c942fc1382fa1c773: Neither blocks nor log segments found. Creating new log.
I20260811 12:02:35.083513 12086 log.cc:824] T 00000000000000000000000000000000 P 186edf6d1f844a7c942fc1382fa1c773: Log is configured to *not* fsync() on all Append() calls
I20260811 12:02:35.086814 12086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 186edf6d1f844a7c942fc1382fa1c773: No bootstrap required, opened a new log
I20260811 12:02:35.087082 12097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:35.064722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34678 (local address 127.9.221.190:43507)
0811 12:02:35.065584 (+ 862us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:35.065598 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:35.065646 (+ 48us) server_negotiation.cc:407] Connection header received
0811 12:02:35.067912 (+ 2266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:35.067938 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:35.068435 (+ 497us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:35.068861 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:35.070097 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:35.072199 (+ 2102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:35.073349 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:35.074354 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:35.074534 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:35.078475 (+ 3941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:35.078485 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:35.078503 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:35.078552 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:35.082634 (+ 4082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:35.083445 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:35.083454 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:35.083462 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:35.083725 (+ 263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:35.084145 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:35.084153 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:35.084158 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:35.084685 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:35.084985 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:35.085357 (+ 372us) server_negotiation.cc:299] Negotiation successful
0811 12:02:35.085766 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":406,"threads_started":1}
I20260811 12:02:35.094806 12086 raft_consensus.cc:348] T 00000000000000000000000000000000 P 186edf6d1f844a7c942fc1382fa1c773 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186edf6d1f844a7c942fc1382fa1c773" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 43507 } }
I20260811 12:02:35.095117 12086 raft_consensus.cc:374] T 00000000000000000000000000000000 P 186edf6d1f844a7c942fc1382fa1c773 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:02:35.095194 12086 raft_consensus.cc:729] T 00000000000000000000000000000000 P 186edf6d1f844a7c942fc1382fa1c773 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 186edf6d1f844a7c942fc1382fa1c773, State: Initialized, Role: FOLLOWER
I20260811 12:02:35.095777 12086 consensus_queue.cc:260] T 00000000000000000000000000000000 P 186edf6d1f844a7c942fc1382fa1c773 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186edf6d1f844a7c942fc1382fa1c773" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 43507 } }
I20260811 12:02:35.095978 12086 raft_consensus.cc:388] T 00000000000000000000000000000000 P 186edf6d1f844a7c942fc1382fa1c773 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:02:35.096076 12086 raft_consensus.cc:482] T 00000000000000000000000000000000 P 186edf6d1f844a7c942fc1382fa1c773 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:02:35.096256 12086 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 186edf6d1f844a7c942fc1382fa1c773 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:02:35.098544 12086 raft_consensus.cc:504] T 00000000000000000000000000000000 P 186edf6d1f844a7c942fc1382fa1c773 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186edf6d1f844a7c942fc1382fa1c773" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 43507 } }
I20260811 12:02:35.098987 12086 leader_election.cc:302] T 00000000000000000000000000000000 P 186edf6d1f844a7c942fc1382fa1c773 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 186edf6d1f844a7c942fc1382fa1c773; no voters:
I20260811 12:02:35.099633 12086 leader_election.cc:288] T 00000000000000000000000000000000 P 186edf6d1f844a7c942fc1382fa1c773 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:02:35.099727 12099 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 186edf6d1f844a7c942fc1382fa1c773 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:02:35.100711 12099 raft_consensus.cc:686] T 00000000000000000000000000000000 P 186edf6d1f844a7c942fc1382fa1c773 [term 1 LEADER]: Becoming Leader. State: Replica: 186edf6d1f844a7c942fc1382fa1c773, State: Running, Role: LEADER
I20260811 12:02:35.101346 12099 consensus_queue.cc:237] T 00000000000000000000000000000000 P 186edf6d1f844a7c942fc1382fa1c773 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186edf6d1f844a7c942fc1382fa1c773" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 43507 } }
I20260811 12:02:35.101996 12086 sys_catalog.cc:564] T 00000000000000000000000000000000 P 186edf6d1f844a7c942fc1382fa1c773 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:02:35.104208 12100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 186edf6d1f844a7c942fc1382fa1c773 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "186edf6d1f844a7c942fc1382fa1c773" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186edf6d1f844a7c942fc1382fa1c773" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 43507 } } }
I20260811 12:02:35.104161 12101 sys_catalog.cc:455] T 00000000000000000000000000000000 P 186edf6d1f844a7c942fc1382fa1c773 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 186edf6d1f844a7c942fc1382fa1c773. Latest consensus state: current_term: 1 leader_uuid: "186edf6d1f844a7c942fc1382fa1c773" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186edf6d1f844a7c942fc1382fa1c773" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 43507 } } }
I20260811 12:02:35.104691 12100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 186edf6d1f844a7c942fc1382fa1c773 [sys.catalog]: This master's current role is: LEADER
I20260811 12:02:35.104748 12101 sys_catalog.cc:458] T 00000000000000000000000000000000 P 186edf6d1f844a7c942fc1382fa1c773 [sys.catalog]: This master's current role is: LEADER
I20260811 12:02:35.112003 12107 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:02:35.116364 12107 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:02:35.124583 12107 catalog_manager.cc:1347] Generated new cluster ID: 4f9ff60beb49458f8a01f2943d806e86
I20260811 12:02:35.124703 12107 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:02:35.143682 12107 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:02:35.144927 12107 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:02:35.154322 12107 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 186edf6d1f844a7c942fc1382fa1c773: Generated new TSK 0
I20260811 12:02:35.155092 12107 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.221.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab.
Entry for principal kudu/127.9.221.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab.
I20260811 12:02:35.208586 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.129:0
--local_ip_for_outbound_sockets=127.9.221.129
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:43507
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab
--principal=kudu/127.9.221.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:34173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:02:35.572752 12132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:02:35.573019 12132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:02:35.573114 12132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:02:35.580180 12132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:02:35.580312 12132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:02:35.580392 12132 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 12:02:35.580471 12132 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.129
I20260811 12:02:35.590041 12132 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:34173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab
--principal=kudu/127.9.221.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.221.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.221.190:43507
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.129
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:02:35.591269 12132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:02:35.592603 12132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:02:35.606096 12142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:35.606118 12146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:35.606318 12143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:02:35.607402 12132 server_base.cc:1034] running on GCE node
I20260811 12:02:35.608117 12132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:02:35.609043 12132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:02:35.610280 12132 hybrid_clock.cc:648] HybridClock initialized: now 1786449755610199 us; error 58 us; skew 500 ppm
I20260811 12:02:35.610579 12132 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:02:35 dist-test-slave-mjjr krb5kdc[11933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449755, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:02:35.613718 12132 init.cc:377] Logged in from keytab as kudu/127.9.221.129@KRBTEST.COM (short username kudu)
I20260811 12:02:35.615067 12132 webserver.cc:460] Webserver started at http://127.9.221.129:44345/ using document root <none> and password file <none>
I20260811 12:02:35.615792 12132 fs_manager.cc:359] Metadata directory not provided
I20260811 12:02:35.615882 12132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:02:35.616158 12132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:02:35.619244 12132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-0/data/instance:
uuid: "ad4fc396f9d0488095791430ce88e06a"
format_stamp: "Formatted at 2026-08-11 12:02:35 on dist-test-slave-mjjr"
server_key: "980824c2dce736c9d926496e3dfbdc58"
server_key_iv: "e7ac805f2e74cb4cb92ee7ac0911ab21"
server_key_version: "encryptionkey@0"
I20260811 12:02:35.620085 12132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance:
uuid: "ad4fc396f9d0488095791430ce88e06a"
format_stamp: "Formatted at 2026-08-11 12:02:35 on dist-test-slave-mjjr"
server_key: "980824c2dce736c9d926496e3dfbdc58"
server_key_iv: "e7ac805f2e74cb4cb92ee7ac0911ab21"
server_key_version: "encryptionkey@0"
I20260811 12:02:35.625114 12132 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 12:02:35.628444 12158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:35.629714 12132 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:02:35.629889 12132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-0/wal
uuid: "ad4fc396f9d0488095791430ce88e06a"
format_stamp: "Formatted at 2026-08-11 12:02:35 on dist-test-slave-mjjr"
server_key: "980824c2dce736c9d926496e3dfbdc58"
server_key_iv: "e7ac805f2e74cb4cb92ee7ac0911ab21"
server_key_version: "encryptionkey@0"
I20260811 12:02:35.630105 12132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:02:35.647728 12132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:02:35.651448 12132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:02:35.651767 12132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:02:35.654455 12132 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:02:35.654563 12132 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:35.654732 12132 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:02:35.654799 12132 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 12:02:35 dist-test-slave-mjjr krb5kdc[11933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449755, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:02:35.678231 12176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:35.655378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:53041 (local address 127.9.221.190:43507)
0811 12:02:35.655891 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:35.655895 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:35.657086 (+ 1191us) server_negotiation.cc:407] Connection header received
0811 12:02:35.659150 (+ 2064us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:35.659156 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:35.659252 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:35.659373 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:35.661657 (+ 2284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:35.662680 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:35.665543 (+ 2863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:35.665944 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:35.666083 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:35.670541 (+ 4458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:35.670545 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:35.670549 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:35.670581 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:35.674517 (+ 3936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:35.675503 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:35.675507 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:35.675509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:35.675676 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:35.676311 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:35.676315 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:35.676317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:35.676739 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:35.676903 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:35.677622 (+ 719us) server_negotiation.cc:299] Negotiation successful
0811 12:02:35.677849 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":297,"threads_started":1}
I20260811 12:02:35.679207 12177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:35.656088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:43507 (local address 127.9.221.129:53041)
0811 12:02:35.656914 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:35.656973 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:35.658878 (+ 1905us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:35.659628 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:35.659674 (+ 46us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:35.660476 (+ 802us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:02:35.661464 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:35.661484 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:35.662903 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:35.662907 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:35.665389 (+ 2482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:35.665409 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:35.666216 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:35.666228 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:35.666440 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:35.667108 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:02:35.667133 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:02:35.670291 (+ 3158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:02:35.674673 (+ 4382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:35.674679 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:35.674691 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:35.675307 (+ 616us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:35.675819 (+ 512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:35.675825 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:35.675828 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:35.676156 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:35.676883 (+ 727us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:02:35.676890 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:02:35.677245 (+ 355us) client_negotiation.cc:769] Sending connection context
0811 12:02:35.677513 (+ 268us) client_negotiation.cc:240] Negotiation successful
0811 12:02:35.678022 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":336,"threads_started":1}
I20260811 12:02:35.700333 12176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:35.685627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:55723 (local address 127.9.221.190:43507)
0811 12:02:35.685817 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:35.685825 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:35.686697 (+ 872us) server_negotiation.cc:407] Connection header received
0811 12:02:35.686986 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:35.686994 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:35.687078 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:35.687153 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:35.688783 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:35.689863 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:35.691900 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:35.692318 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:35.692473 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:35.694203 (+ 1730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:35.694211 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:35.694220 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:35.694249 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:35.697514 (+ 3265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:35.698200 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:35.698206 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:35.698210 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:35.698338 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:35.698744 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:35.698750 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:35.698754 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:35.699096 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:35.699253 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:35.699721 (+ 468us) server_negotiation.cc:299] Negotiation successful
0811 12:02:35.699947 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 12:02:35.700518 12275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:35.685652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:43507 (local address 127.9.221.129:55723)
0811 12:02:35.686590 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:35.686606 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:35.686789 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:35.687319 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:35.687322 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:35.688072 (+ 750us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:02:35.688634 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:35.688646 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:35.690036 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:35.690041 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:35.691735 (+ 1694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:35.691758 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:35.692508 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:35.692515 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:35.692688 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:35.693298 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:02:35.693316 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:02:35.694019 (+ 703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:02:35.697655 (+ 3636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:35.697660 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:35.697664 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:35.698080 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:35.698455 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:35.698462 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:35.698466 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:35.698614 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:35.699219 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:02:35.699224 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:02:35.699403 (+ 179us) client_negotiation.cc:769] Sending connection context
0811 12:02:35.699581 (+ 178us) client_negotiation.cc:240] Negotiation successful
0811 12:02:35.700032 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":787,"thread_start_us":519,"threads_started":1}
I20260811 12:02:35.712721 12132 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.129:33027
I20260811 12:02:35.712745 12369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.129:33027 every 8 connection(s)
I20260811 12:02:35.713958 12132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
I20260811 12:02:35.716159 12370 heartbeater.cc:347] Connected to a master server at 127.9.221.190:43507
I20260811 12:02:35.716432 12370 heartbeater.cc:464] Registering TS with master...
I20260811 12:02:35.717088 12370 heartbeater.cc:511] Master 127.9.221.190:43507 requested a full tablet report, sending...
I20260811 12:02:35.719122 12015 ts_manager.cc:194] Registered new tserver with Master: ad4fc396f9d0488095791430ce88e06a (127.9.221.129:33027)
I20260811 12:02:35.720569 12015 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.129@KRBTEST.COM'} at 127.9.221.129:53041
I20260811 12:02:35.722529 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 12132
I20260811 12:02:35.722752 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance
I20260811 12:02:35.723381 10102 external_mini_cluster.cc:1428] Setting key b2220ee8f6cd1ce3f30c634417d1f672
WARNING: no policy specified for kudu/127.9.221.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab.
Entry for principal kudu/127.9.221.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab.
I20260811 12:02:35.762741 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.130:0
--local_ip_for_outbound_sockets=127.9.221.130
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:43507
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab
--principal=kudu/127.9.221.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:34173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:02:36.089228 12375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:02:36.089514 12375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:02:36.089689 12375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:02:36.097244 12375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:02:36.097389 12375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:02:36.097534 12375 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 12:02:36.097679 12375 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.130
I20260811 12:02:36.107555 12375 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:34173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab
--principal=kudu/127.9.221.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.221.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.221.190:43507
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.130
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:02:36.108870 12375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:02:36.110194 12375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:02:36.123679 12394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:36.123713 12398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:36.123822 12393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:02:36.124579 12375 server_base.cc:1034] running on GCE node
I20260811 12:02:36.125201 12375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:02:36.126075 12375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:02:36.127282 12375 hybrid_clock.cc:648] HybridClock initialized: now 1786449756127227 us; error 35 us; skew 500 ppm
I20260811 12:02:36.127578 12375 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:02:36 dist-test-slave-mjjr krb5kdc[11933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449756, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:02:36.130635 12375 init.cc:377] Logged in from keytab as kudu/127.9.221.130@KRBTEST.COM (short username kudu)
I20260811 12:02:36.132143 12375 webserver.cc:460] Webserver started at http://127.9.221.130:43921/ using document root <none> and password file <none>
I20260811 12:02:36.132827 12375 fs_manager.cc:359] Metadata directory not provided
I20260811 12:02:36.132964 12375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:02:36.133335 12375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:02:36.136271 12375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-1/data/instance:
uuid: "681003820cda4640af8791856d6f2992"
format_stamp: "Formatted at 2026-08-11 12:02:36 on dist-test-slave-mjjr"
server_key: "a81b7ed09b439fceb07f68371733f860"
server_key_iv: "4394c8fad0681e23041c6dcf63ab2e73"
server_key_version: "encryptionkey@0"
I20260811 12:02:36.137054 12375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance:
uuid: "681003820cda4640af8791856d6f2992"
format_stamp: "Formatted at 2026-08-11 12:02:36 on dist-test-slave-mjjr"
server_key: "a81b7ed09b439fceb07f68371733f860"
server_key_iv: "4394c8fad0681e23041c6dcf63ab2e73"
server_key_version: "encryptionkey@0"
I20260811 12:02:36.141996 12375 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 12:02:36.145649 12408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:36.146915 12375 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:02:36.147091 12375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-1/wal
uuid: "681003820cda4640af8791856d6f2992"
format_stamp: "Formatted at 2026-08-11 12:02:36 on dist-test-slave-mjjr"
server_key: "a81b7ed09b439fceb07f68371733f860"
server_key_iv: "4394c8fad0681e23041c6dcf63ab2e73"
server_key_version: "encryptionkey@0"
I20260811 12:02:36.147310 12375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:02:36.166915 12375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:02:36.170723 12375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:02:36.170989 12375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:02:36.173730 12375 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:02:36.173840 12375 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:36.173983 12375 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:02:36.174073 12375 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 12:02:36 dist-test-slave-mjjr krb5kdc[11933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449756, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.130@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:02:36.197198 12176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:36.174110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:59597 (local address 127.9.221.190:43507)
0811 12:02:36.174384 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:36.174393 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:36.175896 (+ 1503us) server_negotiation.cc:407] Connection header received
0811 12:02:36.178083 (+ 2187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:36.178091 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:36.178190 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:36.178328 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:36.180715 (+ 2387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:36.181896 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:36.184579 (+ 2683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:36.185019 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:36.185185 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:36.190449 (+ 5264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:36.190455 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:36.190459 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:36.190492 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:36.193931 (+ 3439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:36.194643 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:36.194647 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:36.194650 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:36.194761 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:36.195235 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:36.195242 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:36.195249 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:36.195517 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:36.195709 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:36.196630 (+ 921us) server_negotiation.cc:299] Negotiation successful
0811 12:02:36.196844 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260811 12:02:36.198187 12422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:36.174856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:43507 (local address 127.9.221.130:59597)
0811 12:02:36.175749 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:36.175786 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:36.177681 (+ 1895us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:36.178668 (+ 987us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:36.178678 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:36.179472 (+ 794us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:02:36.180445 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:36.180475 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:36.182045 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:36.182055 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:36.184415 (+ 2360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:36.184430 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:36.185315 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:36.185323 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:36.185658 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:36.186487 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:02:36.186557 (+ 70us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:02:36.190238 (+ 3681us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:02:36.194071 (+ 3833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:36.194079 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:36.194092 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:36.194506 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:36.194889 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:36.194894 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:36.194898 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:36.195110 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:36.195679 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:02:36.195687 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:02:36.196135 (+ 448us) client_negotiation.cc:769] Sending connection context
0811 12:02:36.196553 (+ 418us) client_negotiation.cc:240] Negotiation successful
0811 12:02:36.197022 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":400,"threads_started":1}
I20260811 12:02:36.222173 12176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:36.204153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:60851 (local address 127.9.221.190:43507)
0811 12:02:36.204342 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:36.204350 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:36.205630 (+ 1280us) server_negotiation.cc:407] Connection header received
0811 12:02:36.205843 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:36.205852 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:36.205945 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:36.206065 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:36.207629 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:36.208935 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:36.211388 (+ 2453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:36.212207 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:36.212345 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:36.214595 (+ 2250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:36.214601 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:36.214607 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:36.214648 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:36.218701 (+ 4053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:36.219752 (+ 1051us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:36.219761 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:36.219766 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:36.219991 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:36.220375 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:36.220380 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:36.220383 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:36.220847 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:36.221007 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:36.221511 (+ 504us) server_negotiation.cc:299] Negotiation successful
0811 12:02:36.221757 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 12:02:36.222201 12375 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.130:43509
I20260811 12:02:36.222221 12605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.130:43509 every 8 connection(s)
I20260811 12:02:36.222383 12528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:36.204747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:43507 (local address 127.9.221.130:60851)
0811 12:02:36.205496 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:36.205526 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:36.205694 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:36.206116 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:36.206121 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:36.206940 (+ 819us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:02:36.207468 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:36.207483 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:36.209121 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:36.209128 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:36.211205 (+ 2077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:36.211225 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:36.212450 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:36.212459 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:36.212687 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:36.213397 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:02:36.213413 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:02:36.214352 (+ 939us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:02:36.218857 (+ 4505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:36.218868 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:36.218873 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:36.219606 (+ 733us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:36.220144 (+ 538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:36.220148 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:36.220150 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:36.220265 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:36.221010 (+ 745us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:02:36.221014 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:02:36.221204 (+ 190us) client_negotiation.cc:769] Sending connection context
0811 12:02:36.221387 (+ 183us) client_negotiation.cc:240] Negotiation successful
0811 12:02:36.221779 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":444,"threads_started":1}
I20260811 12:02:36.223563 12375 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
I20260811 12:02:36.225986 12607 heartbeater.cc:347] Connected to a master server at 127.9.221.190:43507
I20260811 12:02:36.226423 12607 heartbeater.cc:464] Registering TS with master...
I20260811 12:02:36.227293 12607 heartbeater.cc:511] Master 127.9.221.190:43507 requested a full tablet report, sending...
I20260811 12:02:36.228564 12015 ts_manager.cc:194] Registered new tserver with Master: 681003820cda4640af8791856d6f2992 (127.9.221.130:43509)
I20260811 12:02:36.229346 12015 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.130@KRBTEST.COM'} at 127.9.221.130:59597
I20260811 12:02:36.229743 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 12375
I20260811 12:02:36.229980 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance
I20260811 12:02:36.230443 10102 external_mini_cluster.cc:1428] Setting key 823154fab169b5e49a55421d3d19d24a
WARNING: no policy specified for kudu/127.9.221.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab.
Entry for principal kudu/127.9.221.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab.
I20260811 12:02:36.270154 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.131:0
--local_ip_for_outbound_sockets=127.9.221.131
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:43507
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab
--principal=kudu/127.9.221.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:34173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:02:36.612618 12613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:02:36.612953 12613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:02:36.613132 12613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:02:36.620364 12613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:02:36.620515 12613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:02:36.620733 12613 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 12:02:36.620911 12613 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.131
I20260811 12:02:36.630520 12613 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:34173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab
--principal=kudu/127.9.221.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.221.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.221.190:43507
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.131
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:02:36.631829 12613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:02:36.633165 12613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:02:36.646450 12620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:36.646521 12623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:36.646713 12626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:02:36.647470 12613 server_base.cc:1034] running on GCE node
I20260811 12:02:36.648162 12613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:02:36.649061 12613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:02:36.650312 12613 hybrid_clock.cc:648] HybridClock initialized: now 1786449756650253 us; error 34 us; skew 500 ppm
I20260811 12:02:36.650617 12613 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:02:36 dist-test-slave-mjjr krb5kdc[11933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449756, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:02:36.653836 12613 init.cc:377] Logged in from keytab as kudu/127.9.221.131@KRBTEST.COM (short username kudu)
I20260811 12:02:36.655146 12613 webserver.cc:460] Webserver started at http://127.9.221.131:37133/ using document root <none> and password file <none>
I20260811 12:02:36.655896 12613 fs_manager.cc:359] Metadata directory not provided
I20260811 12:02:36.655987 12613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:02:36.656247 12613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:02:36.659106 12613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-2/data/instance:
uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8"
format_stamp: "Formatted at 2026-08-11 12:02:36 on dist-test-slave-mjjr"
server_key: "cd76b9cbf67bca5d47da1eea00e2b77e"
server_key_iv: "7e6598a22d9ad7802f1165bdcb978f19"
server_key_version: "encryptionkey@0"
I20260811 12:02:36.659981 12613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance:
uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8"
format_stamp: "Formatted at 2026-08-11 12:02:36 on dist-test-slave-mjjr"
server_key: "cd76b9cbf67bca5d47da1eea00e2b77e"
server_key_iv: "7e6598a22d9ad7802f1165bdcb978f19"
server_key_version: "encryptionkey@0"
I20260811 12:02:36.664923 12613 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 12:02:36.668267 12637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:36.669497 12613 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:02:36.669675 12613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-2/wal
uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8"
format_stamp: "Formatted at 2026-08-11 12:02:36 on dist-test-slave-mjjr"
server_key: "cd76b9cbf67bca5d47da1eea00e2b77e"
server_key_iv: "7e6598a22d9ad7802f1165bdcb978f19"
server_key_version: "encryptionkey@0"
I20260811 12:02:36.669898 12613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:02:36.695434 12613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:02:36.699412 12613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:02:36.699791 12613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:02:36.702831 12613 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:02:36.702960 12613 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:36.703107 12613 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:02:36.703183 12613 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 12:02:36 dist-test-slave-mjjr krb5kdc[11933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449756, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.131@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:02:36.723204 12370 heartbeater.cc:503] Master 127.9.221.190:43507 was elected leader, sending a full tablet report...
I20260811 12:02:36.732437 12653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:36.703803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:43507 (local address 127.9.221.131:56417)
0811 12:02:36.704751 (+ 948us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:36.704807 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:36.707708 (+ 2901us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:36.708580 (+ 872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:36.708611 (+ 31us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:36.709530 (+ 919us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:02:36.710516 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:36.710546 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:36.711622 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:36.711631 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:36.714597 (+ 2966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:36.714620 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:36.716253 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:36.716260 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:36.716647 (+ 387us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:36.717572 (+ 925us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:02:36.717612 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:02:36.721502 (+ 3890us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:02:36.726511 (+ 5009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:36.726524 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:36.726545 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:36.727288 (+ 743us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:36.727866 (+ 578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:36.727872 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:36.727875 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:36.728177 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:36.728984 (+ 807us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:02:36.728993 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:02:36.729528 (+ 535us) client_negotiation.cc:769] Sending connection context
0811 12:02:36.729899 (+ 371us) client_negotiation.cc:240] Negotiation successful
0811 12:02:36.730555 (+ 656us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"thread_start_us":395,"threads_started":1}
I20260811 12:02:36.732789 12176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:36.703190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:56417 (local address 127.9.221.190:43507)
0811 12:02:36.703424 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:36.703434 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:36.704963 (+ 1529us) server_negotiation.cc:407] Connection header received
0811 12:02:36.708047 (+ 3084us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:36.708053 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:36.708139 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:36.708253 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:36.710719 (+ 2466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:36.711447 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:36.714841 (+ 3394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:36.715744 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:36.715904 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:36.721772 (+ 5868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:36.721782 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:36.721790 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:36.721852 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:36.726346 (+ 4494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:36.727466 (+ 1120us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:36.727474 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:36.727479 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:36.727688 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:36.728370 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:36.728377 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:36.728382 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:36.728804 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:36.731829 (+ 3025us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:36.732070 (+ 241us) server_negotiation.cc:299] Negotiation successful
0811 12:02:36.732381 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 12:02:36.752561 12613 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.131:39597
I20260811 12:02:36.752566 12828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.131:39597 every 8 connection(s)
I20260811 12:02:36.754359 12613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
I20260811 12:02:36.758607 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 12613
I20260811 12:02:36.758937 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance
I20260811 12:02:36.759742 10102 external_mini_cluster.cc:1428] Setting key e75c93e1dc51e0776df034c02ac89d54
I20260811 12:02:36.761325 12830 heartbeater.cc:347] Connected to a master server at 127.9.221.190:43507
I20260811 12:02:36.766407 12830 heartbeater.cc:464] Registering TS with master...
I20260811 12:02:36.767431 12830 heartbeater.cc:511] Master 127.9.221.190:43507 requested a full tablet report, sending...
I20260811 12:02:36.768982 12176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:36.740650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:53013 (local address 127.9.221.190:43507)
0811 12:02:36.740819 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:36.740823 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:36.741505 (+ 682us) server_negotiation.cc:407] Connection header received
0811 12:02:36.741713 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:36.741722 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:36.741822 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:36.741936 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:36.743442 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:36.744634 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:36.747159 (+ 2525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:36.747966 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:36.748136 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:36.750865 (+ 2729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:36.750873 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:36.750879 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:36.750922 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:36.754847 (+ 3925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:36.758429 (+ 3582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:36.758437 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:36.758444 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:36.758615 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:36.767293 (+ 8678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:36.767300 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:36.767304 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:36.767735 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:36.767898 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:36.768356 (+ 458us) server_negotiation.cc:299] Negotiation successful
0811 12:02:36.768603 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 12:02:36.769069 12778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:36.740795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:43507 (local address 127.9.221.131:53013)
0811 12:02:36.741337 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:36.741353 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:36.741554 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:36.742001 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:36.742008 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:36.742737 (+ 729us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:02:36.743260 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:36.743275 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:36.744806 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:36.744814 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:36.746968 (+ 2154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:36.746986 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:36.748206 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:36.748219 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:36.748466 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:36.749309 (+ 843us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:02:36.749337 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:02:36.750598 (+ 1261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:02:36.755039 (+ 4441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:36.755045 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:36.755049 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:36.755766 (+ 717us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:36.766997 (+ 11231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:36.767001 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:36.767004 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:36.767146 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:36.767856 (+ 710us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:02:36.767861 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:02:36.768068 (+ 207us) client_negotiation.cc:769] Sending connection context
0811 12:02:36.768251 (+ 183us) client_negotiation.cc:240] Negotiation successful
0811 12:02:36.768604 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":312,"threads_started":1}
I20260811 12:02:36.776031 12015 ts_manager.cc:194] Registered new tserver with Master: 3bfe3a8d9f9d4ee787e8e0d39835f2e8 (127.9.221.131:39597)
I20260811 12:02:36.776877 12015 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.131@KRBTEST.COM'} at 127.9.221.131:56417
I20260811 12:02:36.779057 10102 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 12:02:36 dist-test-slave-mjjr krb5kdc[11933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449756, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 12:02:36 dist-test-slave-mjjr krb5kdc[11933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449756, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:02:36.836468 12176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:36.819004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51854 (local address 127.9.221.190:43507)
0811 12:02:36.819243 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:36.819252 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:36.819495 (+ 243us) server_negotiation.cc:407] Connection header received
0811 12:02:36.819680 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:36.819688 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:36.819795 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:36.819943 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:36.821435 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:36.822856 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:36.824192 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:36.825084 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:36.825250 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:36.829337 (+ 4087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:36.829344 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:36.829352 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:36.829402 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:36.833361 (+ 3959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:36.834022 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:36.834030 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:36.834036 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:36.834194 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:36.834595 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:36.834603 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:36.834608 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:36.835037 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:36.835392 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:36.835632 (+ 240us) server_negotiation.cc:299] Negotiation successful
0811 12:02:36.835947 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 12:02:36.841233 12015 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51854:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:02:36.855786 12015 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:02:36.886536 12848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:36.872125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.131:39597 (local address 127.0.0.1:35782)
0811 12:02:36.874774 (+ 2649us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:36.874815 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:36.875041 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:36.876271 (+ 1230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:36.876281 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:36.876319 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:02:36.876779 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:36.876795 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:36.879328 (+ 2533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:36.879336 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:36.882455 (+ 3119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:36.882475 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:36.885309 (+ 2834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:36.885316 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:36.885558 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:36.885622 (+ 64us) client_negotiation.cc:769] Sending connection context
0811 12:02:36.885743 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 12:02:36.885998 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2360,"thread_start_us":12518,"threads_started":1}
I20260811 12:02:36.888610 12850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:36.873248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35782 (local address 127.9.221.131:39597)
0811 12:02:36.874327 (+ 1079us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:36.874336 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:36.875237 (+ 901us) server_negotiation.cc:407] Connection header received
0811 12:02:36.875354 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:36.875364 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:36.875632 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:36.875871 (+ 239us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:02:36.877083 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:36.879129 (+ 2046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:36.882698 (+ 3569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:36.884692 (+ 1994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:36.884862 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:36.885152 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:36.887237 (+ 2085us) server_negotiation.cc:299] Negotiation successful
0811 12:02:36.888165 (+ 928us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":872,"thread_start_us":979,"threads_started":1}
I20260811 12:02:36.896651 12847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:36.872764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:33027 (local address 127.0.0.1:60714)
0811 12:02:36.884350 (+ 11586us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:36.884374 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:36.884547 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:36.886583 (+ 2036us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:36.886587 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:36.886618 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:02:36.887307 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:36.887325 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:36.890761 (+ 3436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:36.890766 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:36.893549 (+ 2783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:36.893570 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:36.895770 (+ 2200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:36.895776 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:36.895960 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:36.895993 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 12:02:36.896091 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 12:02:36.896226 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11423,"mutex_wait_us":1,"thread_start_us":953,"threads_started":1}
I20260811 12:02:36.897495 12849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:36.873128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60714 (local address 127.9.221.129:33027)
0811 12:02:36.873734 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:36.873743 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:36.885858 (+ 12115us) server_negotiation.cc:407] Connection header received
0811 12:02:36.886017 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:36.886028 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:36.886353 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:36.886589 (+ 236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:02:36.888785 (+ 2196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:36.890575 (+ 1790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:36.893781 (+ 3206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:36.895222 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:36.895350 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:36.895572 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:36.897020 (+ 1448us) server_negotiation.cc:299] Negotiation successful
0811 12:02:36.897204 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":234,"threads_started":1}
I20260811 12:02:36.899369 12712 tablet_service.cc:1467] Processing CreateTablet for tablet 4cbe8394fd564eef85a68aa11e7b6810 (DEFAULT_TABLE table=test-table [id=18a1efef4d0241a393ba9504f20fe9ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:02:36.901136 12249 tablet_service.cc:1467] Processing CreateTablet for tablet 4cbe8394fd564eef85a68aa11e7b6810 (DEFAULT_TABLE table=test-table [id=18a1efef4d0241a393ba9504f20fe9ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:02:36.901764 12712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cbe8394fd564eef85a68aa11e7b6810. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:02:36.903115 12249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cbe8394fd564eef85a68aa11e7b6810. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:02:36.922824 12848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:36.885350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.130:43509 (local address 127.0.0.1:38794)
0811 12:02:36.886861 (+ 1511us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:36.886887 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:36.887017 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:36.891111 (+ 4094us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:36.891119 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:36.891157 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:02:36.892373 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:36.892390 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:36.894582 (+ 2192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:36.894587 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:36.897639 (+ 3052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:36.897659 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:36.921645 (+ 23986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:36.921654 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:36.921977 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:36.922040 (+ 63us) client_negotiation.cc:769] Sending connection context
0811 12:02:36.922126 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 12:02:36.922340 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1355,"mutex_wait_us":857,"thread_start_us":8688,"threads_started":1}
I20260811 12:02:36.924425 12853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:36.885381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38794 (local address 127.9.221.130:43509)
0811 12:02:36.890459 (+ 5078us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:36.890467 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:36.890498 (+ 31us) server_negotiation.cc:407] Connection header received
0811 12:02:36.890608 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:36.890618 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:36.890867 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:36.891092 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:02:36.892575 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:36.894355 (+ 1780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:36.897887 (+ 3532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:36.921393 (+ 23506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:36.923268 (+ 1875us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:36.923574 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:36.923701 (+ 127us) server_negotiation.cc:299] Negotiation successful
0811 12:02:36.923939 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4922,"thread_start_us":4812,"threads_started":1}
I20260811 12:02:36.925712 12855 tablet_bootstrap.cc:492] T 4cbe8394fd564eef85a68aa11e7b6810 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8: Bootstrap starting.
I20260811 12:02:36.929983 12857 tablet_bootstrap.cc:492] T 4cbe8394fd564eef85a68aa11e7b6810 P ad4fc396f9d0488095791430ce88e06a: Bootstrap starting.
I20260811 12:02:36.929517 12500 tablet_service.cc:1467] Processing CreateTablet for tablet 4cbe8394fd564eef85a68aa11e7b6810 (DEFAULT_TABLE table=test-table [id=18a1efef4d0241a393ba9504f20fe9ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:02:36.931604 12855 tablet_bootstrap.cc:654] T 4cbe8394fd564eef85a68aa11e7b6810 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8: Neither blocks nor log segments found. Creating new log.
I20260811 12:02:36.931726 12500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cbe8394fd564eef85a68aa11e7b6810. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:02:36.935151 12855 log.cc:824] T 4cbe8394fd564eef85a68aa11e7b6810 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8: Log is configured to *not* fsync() on all Append() calls
I20260811 12:02:36.938441 12857 tablet_bootstrap.cc:654] T 4cbe8394fd564eef85a68aa11e7b6810 P ad4fc396f9d0488095791430ce88e06a: Neither blocks nor log segments found. Creating new log.
I20260811 12:02:36.940516 12857 log.cc:824] T 4cbe8394fd564eef85a68aa11e7b6810 P ad4fc396f9d0488095791430ce88e06a: Log is configured to *not* fsync() on all Append() calls
I20260811 12:02:36.948428 12855 tablet_bootstrap.cc:492] T 4cbe8394fd564eef85a68aa11e7b6810 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8: No bootstrap required, opened a new log
I20260811 12:02:36.948913 12855 ts_tablet_manager.cc:1397] T 4cbe8394fd564eef85a68aa11e7b6810 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8: Time spent bootstrapping tablet: real 0.023s user 0.014s sys 0.004s
I20260811 12:02:36.951152 12857 tablet_bootstrap.cc:492] T 4cbe8394fd564eef85a68aa11e7b6810 P ad4fc396f9d0488095791430ce88e06a: No bootstrap required, opened a new log
I20260811 12:02:36.951833 12857 ts_tablet_manager.cc:1397] T 4cbe8394fd564eef85a68aa11e7b6810 P ad4fc396f9d0488095791430ce88e06a: Time spent bootstrapping tablet: real 0.022s user 0.007s sys 0.008s
I20260811 12:02:36.963997 12861 tablet_bootstrap.cc:492] T 4cbe8394fd564eef85a68aa11e7b6810 P 681003820cda4640af8791856d6f2992: Bootstrap starting.
I20260811 12:02:36.968232 12857 raft_consensus.cc:348] T 4cbe8394fd564eef85a68aa11e7b6810 P ad4fc396f9d0488095791430ce88e06a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39597 } } peers { permanent_uuid: "ad4fc396f9d0488095791430ce88e06a" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33027 } } peers { permanent_uuid: "681003820cda4640af8791856d6f2992" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43509 } }
I20260811 12:02:36.968834 12857 raft_consensus.cc:374] T 4cbe8394fd564eef85a68aa11e7b6810 P ad4fc396f9d0488095791430ce88e06a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:02:36.968983 12857 raft_consensus.cc:729] T 4cbe8394fd564eef85a68aa11e7b6810 P ad4fc396f9d0488095791430ce88e06a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad4fc396f9d0488095791430ce88e06a, State: Initialized, Role: FOLLOWER
I20260811 12:02:36.969831 12857 consensus_queue.cc:260] T 4cbe8394fd564eef85a68aa11e7b6810 P ad4fc396f9d0488095791430ce88e06a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39597 } } peers { permanent_uuid: "ad4fc396f9d0488095791430ce88e06a" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33027 } } peers { permanent_uuid: "681003820cda4640af8791856d6f2992" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43509 } }
I20260811 12:02:36.970587 12861 tablet_bootstrap.cc:654] T 4cbe8394fd564eef85a68aa11e7b6810 P 681003820cda4640af8791856d6f2992: Neither blocks nor log segments found. Creating new log.
I20260811 12:02:36.970178 12855 raft_consensus.cc:348] T 4cbe8394fd564eef85a68aa11e7b6810 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39597 } } peers { permanent_uuid: "ad4fc396f9d0488095791430ce88e06a" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33027 } } peers { permanent_uuid: "681003820cda4640af8791856d6f2992" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43509 } }
I20260811 12:02:36.971186 12855 raft_consensus.cc:374] T 4cbe8394fd564eef85a68aa11e7b6810 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:02:36.971470 12855 raft_consensus.cc:729] T 4cbe8394fd564eef85a68aa11e7b6810 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bfe3a8d9f9d4ee787e8e0d39835f2e8, State: Initialized, Role: FOLLOWER
I20260811 12:02:36.972515 12855 consensus_queue.cc:260] T 4cbe8394fd564eef85a68aa11e7b6810 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39597 } } peers { permanent_uuid: "ad4fc396f9d0488095791430ce88e06a" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33027 } } peers { permanent_uuid: "681003820cda4640af8791856d6f2992" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43509 } }
I20260811 12:02:36.973126 12861 log.cc:824] T 4cbe8394fd564eef85a68aa11e7b6810 P 681003820cda4640af8791856d6f2992: Log is configured to *not* fsync() on all Append() calls
I20260811 12:02:36.979228 12857 ts_tablet_manager.cc:1428] T 4cbe8394fd564eef85a68aa11e7b6810 P ad4fc396f9d0488095791430ce88e06a: Time spent starting tablet: real 0.027s user 0.011s sys 0.014s
I20260811 12:02:36.981014 12830 heartbeater.cc:503] Master 127.9.221.190:43507 was elected leader, sending a full tablet report...
I20260811 12:02:36.981173 12861 tablet_bootstrap.cc:492] T 4cbe8394fd564eef85a68aa11e7b6810 P 681003820cda4640af8791856d6f2992: No bootstrap required, opened a new log
I20260811 12:02:36.981580 12861 ts_tablet_manager.cc:1397] T 4cbe8394fd564eef85a68aa11e7b6810 P 681003820cda4640af8791856d6f2992: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.012s
I20260811 12:02:36.985327 12855 ts_tablet_manager.cc:1428] T 4cbe8394fd564eef85a68aa11e7b6810 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8: Time spent starting tablet: real 0.036s user 0.017s sys 0.010s
I20260811 12:02:36.995066 12865 raft_consensus.cc:482] T 4cbe8394fd564eef85a68aa11e7b6810 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:02:36.995610 12865 raft_consensus.cc:504] T 4cbe8394fd564eef85a68aa11e7b6810 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39597 } } peers { permanent_uuid: "ad4fc396f9d0488095791430ce88e06a" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33027 } } peers { permanent_uuid: "681003820cda4640af8791856d6f2992" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43509 } }
I20260811 12:02:36.996752 12861 raft_consensus.cc:348] T 4cbe8394fd564eef85a68aa11e7b6810 P 681003820cda4640af8791856d6f2992 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39597 } } peers { permanent_uuid: "ad4fc396f9d0488095791430ce88e06a" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33027 } } peers { permanent_uuid: "681003820cda4640af8791856d6f2992" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43509 } }
I20260811 12:02:36.997474 12861 raft_consensus.cc:374] T 4cbe8394fd564eef85a68aa11e7b6810 P 681003820cda4640af8791856d6f2992 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:02:36.997679 12861 raft_consensus.cc:729] T 4cbe8394fd564eef85a68aa11e7b6810 P 681003820cda4640af8791856d6f2992 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 681003820cda4640af8791856d6f2992, State: Initialized, Role: FOLLOWER
I20260811 12:02:36.998692 12861 consensus_queue.cc:260] T 4cbe8394fd564eef85a68aa11e7b6810 P 681003820cda4640af8791856d6f2992 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39597 } } peers { permanent_uuid: "ad4fc396f9d0488095791430ce88e06a" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33027 } } peers { permanent_uuid: "681003820cda4640af8791856d6f2992" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43509 } }
I20260811 12:02:37.001006 12607 heartbeater.cc:503] Master 127.9.221.190:43507 was elected leader, sending a full tablet report...
I20260811 12:02:37.002202 12861 ts_tablet_manager.cc:1428] T 4cbe8394fd564eef85a68aa11e7b6810 P 681003820cda4640af8791856d6f2992: Time spent starting tablet: real 0.020s user 0.020s sys 0.000s
I20260811 12:02:36.998811 12865 leader_election.cc:288] T 4cbe8394fd564eef85a68aa11e7b6810 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad4fc396f9d0488095791430ce88e06a (127.9.221.129:33027), 681003820cda4640af8791856d6f2992 (127.9.221.130:43509)
W20260811 12:02:37.010988 12831 tablet.cc:2369] T 4cbe8394fd564eef85a68aa11e7b6810 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:02:37.013046 12653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:36.999797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:33027 (local address 127.9.221.131:53589)
0811 12:02:37.002260 (+ 2463us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:37.002281 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:37.002435 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:37.002970 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:37.002979 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:37.003062 (+ 83us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:02:37.003538 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:37.003558 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:37.005663 (+ 2105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:37.005672 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:37.008861 (+ 3189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:37.008886 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:37.011770 (+ 2884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:37.011779 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:37.012216 (+ 437us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:37.012296 (+ 80us) client_negotiation.cc:769] Sending connection context
0811 12:02:37.012417 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 12:02:37.012611 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 12:02:37.013856 12849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:36.999596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:53589 (local address 127.9.221.129:33027)
0811 12:02:37.000153 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:37.000162 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:37.002559 (+ 2397us) server_negotiation.cc:407] Connection header received
0811 12:02:37.002648 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:37.002653 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:37.002793 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:37.002893 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:02:37.003818 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:37.005440 (+ 1622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:37.009119 (+ 3679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:37.011391 (+ 2272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:37.011613 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:37.013511 (+ 1898us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:37.013568 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 12:02:37.013658 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312}
I20260811 12:02:37.015429 12294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cbe8394fd564eef85a68aa11e7b6810" candidate_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad4fc396f9d0488095791430ce88e06a" is_pre_election: true
I20260811 12:02:37.016325 12294 raft_consensus.cc:2454] T 4cbe8394fd564eef85a68aa11e7b6810 P ad4fc396f9d0488095791430ce88e06a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bfe3a8d9f9d4ee787e8e0d39835f2e8 in term 0.
I20260811 12:02:37.017882 12644 leader_election.cc:302] T 4cbe8394fd564eef85a68aa11e7b6810 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bfe3a8d9f9d4ee787e8e0d39835f2e8, ad4fc396f9d0488095791430ce88e06a; no voters:
I20260811 12:02:37.018591 12865 raft_consensus.cc:2781] T 4cbe8394fd564eef85a68aa11e7b6810 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:02:37.019032 12865 raft_consensus.cc:482] T 4cbe8394fd564eef85a68aa11e7b6810 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:02:37.019340 12865 raft_consensus.cc:3037] T 4cbe8394fd564eef85a68aa11e7b6810 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:02:37.023185 12853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:36.999540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:46589 (local address 127.9.221.130:43509)
0811 12:02:36.999984 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:36.999992 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:37.012982 (+ 12990us) server_negotiation.cc:407] Connection header received
0811 12:02:37.013226 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:37.013236 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:37.013347 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:37.013499 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:02:37.014944 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:37.016564 (+ 1620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:37.020545 (+ 3981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:37.021833 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:37.022096 (+ 263us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:37.022192 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:37.022673 (+ 481us) server_negotiation.cc:299] Negotiation successful
0811 12:02:37.022813 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164}
I20260811 12:02:37.023723 12869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:37.000449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.130:43509 (local address 127.9.221.131:46589)
0811 12:02:37.012841 (+ 12392us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:37.012865 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:37.013043 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:37.013763 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:37.013772 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:37.013811 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:02:37.014721 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:37.014741 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:37.016806 (+ 2065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:37.016824 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:37.020340 (+ 3516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:37.020352 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:37.022313 (+ 1961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:37.022320 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:37.022504 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:37.022528 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 12:02:37.022647 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 12:02:37.022781 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12203,"thread_start_us":12698,"threads_started":1}
I20260811 12:02:37.025293 12534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cbe8394fd564eef85a68aa11e7b6810" candidate_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "681003820cda4640af8791856d6f2992" is_pre_election: true
I20260811 12:02:37.025557 12865 raft_consensus.cc:504] T 4cbe8394fd564eef85a68aa11e7b6810 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39597 } } peers { permanent_uuid: "ad4fc396f9d0488095791430ce88e06a" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33027 } } peers { permanent_uuid: "681003820cda4640af8791856d6f2992" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43509 } }
I20260811 12:02:37.026239 12534 raft_consensus.cc:2454] T 4cbe8394fd564eef85a68aa11e7b6810 P 681003820cda4640af8791856d6f2992 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bfe3a8d9f9d4ee787e8e0d39835f2e8 in term 0.
I20260811 12:02:37.026932 12865 leader_election.cc:288] T 4cbe8394fd564eef85a68aa11e7b6810 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [CANDIDATE]: Term 1 election: Requested vote from peers ad4fc396f9d0488095791430ce88e06a (127.9.221.129:33027), 681003820cda4640af8791856d6f2992 (127.9.221.130:43509)
I20260811 12:02:37.027299 12294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cbe8394fd564eef85a68aa11e7b6810" candidate_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad4fc396f9d0488095791430ce88e06a"
I20260811 12:02:37.027609 12294 raft_consensus.cc:3037] T 4cbe8394fd564eef85a68aa11e7b6810 P ad4fc396f9d0488095791430ce88e06a [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:02:37.028406 12534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cbe8394fd564eef85a68aa11e7b6810" candidate_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "681003820cda4640af8791856d6f2992"
I20260811 12:02:37.028702 12534 raft_consensus.cc:3037] T 4cbe8394fd564eef85a68aa11e7b6810 P 681003820cda4640af8791856d6f2992 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:02:37.031925 12534 raft_consensus.cc:2454] T 4cbe8394fd564eef85a68aa11e7b6810 P 681003820cda4640af8791856d6f2992 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bfe3a8d9f9d4ee787e8e0d39835f2e8 in term 1.
I20260811 12:02:37.032593 12640 leader_election.cc:302] T 4cbe8394fd564eef85a68aa11e7b6810 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bfe3a8d9f9d4ee787e8e0d39835f2e8, 681003820cda4640af8791856d6f2992; no voters:
I20260811 12:02:37.032876 12294 raft_consensus.cc:2454] T 4cbe8394fd564eef85a68aa11e7b6810 P ad4fc396f9d0488095791430ce88e06a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bfe3a8d9f9d4ee787e8e0d39835f2e8 in term 1.
I20260811 12:02:37.033352 12865 raft_consensus.cc:2781] T 4cbe8394fd564eef85a68aa11e7b6810 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:02:37.034525 12865 raft_consensus.cc:686] T 4cbe8394fd564eef85a68aa11e7b6810 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [term 1 LEADER]: Becoming Leader. State: Replica: 3bfe3a8d9f9d4ee787e8e0d39835f2e8, State: Running, Role: LEADER
I20260811 12:02:37.035246 12865 consensus_queue.cc:237] T 4cbe8394fd564eef85a68aa11e7b6810 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39597 } } peers { permanent_uuid: "ad4fc396f9d0488095791430ce88e06a" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33027 } } peers { permanent_uuid: "681003820cda4640af8791856d6f2992" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43509 } }
I20260811 12:02:37.041175 12012 catalog_manager.cc:5495] T 4cbe8394fd564eef85a68aa11e7b6810 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3bfe3a8d9f9d4ee787e8e0d39835f2e8 (127.9.221.131). New cstate: current_term: 1 leader_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad4fc396f9d0488095791430ce88e06a" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "681003820cda4640af8791856d6f2992" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43509 } health_report { overall_health: UNKNOWN } } }
I20260811 12:02:37.094054 12850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:37.086918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35796 (local address 127.9.221.131:39597)
0811 12:02:37.087124 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:37.087129 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:37.087533 (+ 404us) server_negotiation.cc:407] Connection header received
0811 12:02:37.088053 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:37.088058 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:37.088144 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:37.088303 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:02:37.088748 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:37.089694 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:37.091419 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:37.092059 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:37.092206 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:37.092784 (+ 578us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:02:37.093366 (+ 582us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:02:37.093544 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:37.093680 (+ 136us) server_negotiation.cc:299] Negotiation successful
0811 12:02:37.093764 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 12:02:37.108750 12294 raft_consensus.cc:1261] T 4cbe8394fd564eef85a68aa11e7b6810 P ad4fc396f9d0488095791430ce88e06a [term 1 FOLLOWER]: Refusing update from remote peer 3bfe3a8d9f9d4ee787e8e0d39835f2e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:02:37.108884 12534 raft_consensus.cc:1261] T 4cbe8394fd564eef85a68aa11e7b6810 P 681003820cda4640af8791856d6f2992 [term 1 FOLLOWER]: Refusing update from remote peer 3bfe3a8d9f9d4ee787e8e0d39835f2e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:02:37.110076 12870 consensus_queue.cc:1035] T 4cbe8394fd564eef85a68aa11e7b6810 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad4fc396f9d0488095791430ce88e06a" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:02:37.110669 12865 consensus_queue.cc:1035] T 4cbe8394fd564eef85a68aa11e7b6810 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "681003820cda4640af8791856d6f2992" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:02:37.131166 12876 mvcc.cc:205] Tried to move back new op lower bound from 7317298205098123264 to 7317298204821471232. Current Snapshot: MvccSnapshot[applied={T|T < 7317298205098123264}]
I20260811 12:02:37.136689 12878 mvcc.cc:205] Tried to move back new op lower bound from 7317298205098123264 to 7317298204821471232. Current Snapshot: MvccSnapshot[applied={T|T < 7317298205098123264}]
I20260811 12:02:37.162001 12012 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51854:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:02:37.162406 12012 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51854:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:02:37.166849 12012 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 186edf6d1f844a7c942fc1382fa1c773: Sending DeleteTablet for 3 replicas of tablet 4cbe8394fd564eef85a68aa11e7b6810
I20260811 12:02:37.168852 12249 tablet_service.cc:1514] Processing DeleteTablet for tablet 4cbe8394fd564eef85a68aa11e7b6810 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:02:37 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:60714
I20260811 12:02:37.168882 12500 tablet_service.cc:1514] Processing DeleteTablet for tablet 4cbe8394fd564eef85a68aa11e7b6810 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:02:37 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:38794
I20260811 12:02:37.170747 12894 tablet_replica.cc:330] T 4cbe8394fd564eef85a68aa11e7b6810 P 681003820cda4640af8791856d6f2992: stopping tablet replica
I20260811 12:02:37.170615 12012 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51854:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:02:37.171253 12012 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:02:37.171236 12894 raft_consensus.cc:2229] T 4cbe8394fd564eef85a68aa11e7b6810 P 681003820cda4640af8791856d6f2992 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:02:37.172441 12894 raft_consensus.cc:2258] T 4cbe8394fd564eef85a68aa11e7b6810 P 681003820cda4640af8791856d6f2992 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:02:37.172859 12895 tablet_replica.cc:330] T 4cbe8394fd564eef85a68aa11e7b6810 P ad4fc396f9d0488095791430ce88e06a: stopping tablet replica
I20260811 12:02:37.173161 12712 tablet_service.cc:1514] Processing DeleteTablet for tablet 4cbe8394fd564eef85a68aa11e7b6810 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:02:37 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:35782
I20260811 12:02:37.173350 12895 raft_consensus.cc:2229] T 4cbe8394fd564eef85a68aa11e7b6810 P ad4fc396f9d0488095791430ce88e06a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:02:37.174111 12895 raft_consensus.cc:2258] T 4cbe8394fd564eef85a68aa11e7b6810 P ad4fc396f9d0488095791430ce88e06a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:02:37.179766 12895 ts_tablet_manager.cc:1905] T 4cbe8394fd564eef85a68aa11e7b6810 P ad4fc396f9d0488095791430ce88e06a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:02:37.181041 12249 tablet_service.cc:1467] Processing CreateTablet for tablet 26117350248d4571bbf0415710a6db2f (DEFAULT_TABLE table=test-table [id=93f5274b97884b4783bf4329357f1cb1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:02:37.181557 12249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26117350248d4571bbf0415710a6db2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:02:37.181473 12709 tablet_service.cc:1467] Processing CreateTablet for tablet 26117350248d4571bbf0415710a6db2f (DEFAULT_TABLE table=test-table [id=93f5274b97884b4783bf4329357f1cb1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:02:37.182200 12500 tablet_service.cc:1467] Processing CreateTablet for tablet 26117350248d4571bbf0415710a6db2f (DEFAULT_TABLE table=test-table [id=93f5274b97884b4783bf4329357f1cb1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:02:37.182835 12500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26117350248d4571bbf0415710a6db2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:02:37.183830 12709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26117350248d4571bbf0415710a6db2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:02:37.190234 12857 tablet_bootstrap.cc:492] T 26117350248d4571bbf0415710a6db2f P ad4fc396f9d0488095791430ce88e06a: Bootstrap starting.
I20260811 12:02:37.191061 12895 ts_tablet_manager.cc:1918] T 4cbe8394fd564eef85a68aa11e7b6810 P ad4fc396f9d0488095791430ce88e06a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:02:37.191325 12895 log.cc:1196] T 4cbe8394fd564eef85a68aa11e7b6810 P ad4fc396f9d0488095791430ce88e06a: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-0/wal/wals/4cbe8394fd564eef85a68aa11e7b6810
I20260811 12:02:37.191794 12898 tablet_replica.cc:330] T 4cbe8394fd564eef85a68aa11e7b6810 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8: stopping tablet replica
I20260811 12:02:37.192366 12895 ts_tablet_manager.cc:1939] T 4cbe8394fd564eef85a68aa11e7b6810 P ad4fc396f9d0488095791430ce88e06a: Deleting consensus metadata
I20260811 12:02:37.192757 12857 tablet_bootstrap.cc:654] T 26117350248d4571bbf0415710a6db2f P ad4fc396f9d0488095791430ce88e06a: Neither blocks nor log segments found. Creating new log.
I20260811 12:02:37.193647 12894 ts_tablet_manager.cc:1905] T 4cbe8394fd564eef85a68aa11e7b6810 P 681003820cda4640af8791856d6f2992: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:02:37.194648 11985 catalog_manager.cc:4841] TS ad4fc396f9d0488095791430ce88e06a (127.9.221.129:33027): tablet 4cbe8394fd564eef85a68aa11e7b6810 (table test-table [id=18a1efef4d0241a393ba9504f20fe9ba]) successfully deleted
I20260811 12:02:37.201296 12857 tablet_bootstrap.cc:492] T 26117350248d4571bbf0415710a6db2f P ad4fc396f9d0488095791430ce88e06a: No bootstrap required, opened a new log
I20260811 12:02:37.201565 12857 ts_tablet_manager.cc:1397] T 26117350248d4571bbf0415710a6db2f P ad4fc396f9d0488095791430ce88e06a: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260811 12:02:37.203130 12857 raft_consensus.cc:348] T 26117350248d4571bbf0415710a6db2f P ad4fc396f9d0488095791430ce88e06a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad4fc396f9d0488095791430ce88e06a" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33027 } } peers { permanent_uuid: "681003820cda4640af8791856d6f2992" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43509 } } peers { permanent_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39597 } }
I20260811 12:02:37.203543 12857 raft_consensus.cc:374] T 26117350248d4571bbf0415710a6db2f P ad4fc396f9d0488095791430ce88e06a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:02:37.203656 12861 tablet_bootstrap.cc:492] T 26117350248d4571bbf0415710a6db2f P 681003820cda4640af8791856d6f2992: Bootstrap starting.
I20260811 12:02:37.203837 12894 ts_tablet_manager.cc:1918] T 4cbe8394fd564eef85a68aa11e7b6810 P 681003820cda4640af8791856d6f2992: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:02:37.203799 12857 raft_consensus.cc:729] T 26117350248d4571bbf0415710a6db2f P ad4fc396f9d0488095791430ce88e06a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad4fc396f9d0488095791430ce88e06a, State: Initialized, Role: FOLLOWER
I20260811 12:02:37.204093 12894 log.cc:1196] T 4cbe8394fd564eef85a68aa11e7b6810 P 681003820cda4640af8791856d6f2992: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-1/wal/wals/4cbe8394fd564eef85a68aa11e7b6810
I20260811 12:02:37.204288 12857 consensus_queue.cc:260] T 26117350248d4571bbf0415710a6db2f P ad4fc396f9d0488095791430ce88e06a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad4fc396f9d0488095791430ce88e06a" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33027 } } peers { permanent_uuid: "681003820cda4640af8791856d6f2992" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43509 } } peers { permanent_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39597 } }
I20260811 12:02:37.205272 12898 raft_consensus.cc:2229] T 4cbe8394fd564eef85a68aa11e7b6810 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:02:37.206063 12855 tablet_bootstrap.cc:492] T 26117350248d4571bbf0415710a6db2f P 3bfe3a8d9f9d4ee787e8e0d39835f2e8: Bootstrap starting.
I20260811 12:02:37.206310 12894 ts_tablet_manager.cc:1939] T 4cbe8394fd564eef85a68aa11e7b6810 P 681003820cda4640af8791856d6f2992: Deleting consensus metadata
I20260811 12:02:37.207791 12861 tablet_bootstrap.cc:654] T 26117350248d4571bbf0415710a6db2f P 681003820cda4640af8791856d6f2992: Neither blocks nor log segments found. Creating new log.
I20260811 12:02:37.209124 12855 tablet_bootstrap.cc:654] T 26117350248d4571bbf0415710a6db2f P 3bfe3a8d9f9d4ee787e8e0d39835f2e8: Neither blocks nor log segments found. Creating new log.
I20260811 12:02:37.209714 11980 catalog_manager.cc:4841] TS 681003820cda4640af8791856d6f2992 (127.9.221.130:43509): tablet 4cbe8394fd564eef85a68aa11e7b6810 (table test-table [id=18a1efef4d0241a393ba9504f20fe9ba]) successfully deleted
I20260811 12:02:37.210278 12898 raft_consensus.cc:2258] T 4cbe8394fd564eef85a68aa11e7b6810 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:02:37.210326 12857 ts_tablet_manager.cc:1428] T 26117350248d4571bbf0415710a6db2f P ad4fc396f9d0488095791430ce88e06a: Time spent starting tablet: real 0.008s user 0.000s sys 0.003s
I20260811 12:02:37.214468 12855 tablet_bootstrap.cc:492] T 26117350248d4571bbf0415710a6db2f P 3bfe3a8d9f9d4ee787e8e0d39835f2e8: No bootstrap required, opened a new log
I20260811 12:02:37.214429 12861 tablet_bootstrap.cc:492] T 26117350248d4571bbf0415710a6db2f P 681003820cda4640af8791856d6f2992: No bootstrap required, opened a new log
I20260811 12:02:37.214697 12855 ts_tablet_manager.cc:1397] T 26117350248d4571bbf0415710a6db2f P 3bfe3a8d9f9d4ee787e8e0d39835f2e8: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.004s
I20260811 12:02:37.214747 12861 ts_tablet_manager.cc:1397] T 26117350248d4571bbf0415710a6db2f P 681003820cda4640af8791856d6f2992: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.005s
I20260811 12:02:37.214867 12898 ts_tablet_manager.cc:1905] T 4cbe8394fd564eef85a68aa11e7b6810 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:02:37.216037 12855 raft_consensus.cc:348] T 26117350248d4571bbf0415710a6db2f P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad4fc396f9d0488095791430ce88e06a" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33027 } } peers { permanent_uuid: "681003820cda4640af8791856d6f2992" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43509 } } peers { permanent_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39597 } }
I20260811 12:02:37.216328 12855 raft_consensus.cc:374] T 26117350248d4571bbf0415710a6db2f P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:02:37.216259 12861 raft_consensus.cc:348] T 26117350248d4571bbf0415710a6db2f P 681003820cda4640af8791856d6f2992 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad4fc396f9d0488095791430ce88e06a" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33027 } } peers { permanent_uuid: "681003820cda4640af8791856d6f2992" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43509 } } peers { permanent_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39597 } }
I20260811 12:02:37.216552 12855 raft_consensus.cc:729] T 26117350248d4571bbf0415710a6db2f P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bfe3a8d9f9d4ee787e8e0d39835f2e8, State: Initialized, Role: FOLLOWER
I20260811 12:02:37.216575 12861 raft_consensus.cc:374] T 26117350248d4571bbf0415710a6db2f P 681003820cda4640af8791856d6f2992 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260811 12:02:37.216552 12371 tablet.cc:2369] T 26117350248d4571bbf0415710a6db2f P ad4fc396f9d0488095791430ce88e06a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:02:37.216774 12861 raft_consensus.cc:729] T 26117350248d4571bbf0415710a6db2f P 681003820cda4640af8791856d6f2992 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 681003820cda4640af8791856d6f2992, State: Initialized, Role: FOLLOWER
I20260811 12:02:37.217007 12855 consensus_queue.cc:260] T 26117350248d4571bbf0415710a6db2f P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad4fc396f9d0488095791430ce88e06a" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33027 } } peers { permanent_uuid: "681003820cda4640af8791856d6f2992" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43509 } } peers { permanent_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39597 } }
I20260811 12:02:37.217324 12861 consensus_queue.cc:260] T 26117350248d4571bbf0415710a6db2f P 681003820cda4640af8791856d6f2992 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad4fc396f9d0488095791430ce88e06a" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33027 } } peers { permanent_uuid: "681003820cda4640af8791856d6f2992" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43509 } } peers { permanent_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39597 } }
I20260811 12:02:37.218848 12861 ts_tablet_manager.cc:1428] T 26117350248d4571bbf0415710a6db2f P 681003820cda4640af8791856d6f2992: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260811 12:02:37.219082 12855 ts_tablet_manager.cc:1428] T 26117350248d4571bbf0415710a6db2f P 3bfe3a8d9f9d4ee787e8e0d39835f2e8: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
W20260811 12:02:37.227969 12608 tablet.cc:2369] T 26117350248d4571bbf0415710a6db2f P 681003820cda4640af8791856d6f2992: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:02:37.228940 12898 ts_tablet_manager.cc:1918] T 4cbe8394fd564eef85a68aa11e7b6810 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:02:37.229212 12898 log.cc:1196] T 4cbe8394fd564eef85a68aa11e7b6810 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-2/wal/wals/4cbe8394fd564eef85a68aa11e7b6810
I20260811 12:02:37.230098 12898 ts_tablet_manager.cc:1939] T 4cbe8394fd564eef85a68aa11e7b6810 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8: Deleting consensus metadata
I20260811 12:02:37.232203 11980 catalog_manager.cc:4841] TS 3bfe3a8d9f9d4ee787e8e0d39835f2e8 (127.9.221.131:39597): tablet 4cbe8394fd564eef85a68aa11e7b6810 (table test-table [id=18a1efef4d0241a393ba9504f20fe9ba]) successfully deleted
I20260811 12:02:37.483847 12865 raft_consensus.cc:482] T 26117350248d4571bbf0415710a6db2f P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:02:37.484218 12865 raft_consensus.cc:504] T 26117350248d4571bbf0415710a6db2f P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad4fc396f9d0488095791430ce88e06a" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33027 } } peers { permanent_uuid: "681003820cda4640af8791856d6f2992" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43509 } } peers { permanent_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39597 } }
I20260811 12:02:37.485417 12865 leader_election.cc:288] T 26117350248d4571bbf0415710a6db2f P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad4fc396f9d0488095791430ce88e06a (127.9.221.129:33027), 681003820cda4640af8791856d6f2992 (127.9.221.130:43509)
I20260811 12:02:37.485962 12294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26117350248d4571bbf0415710a6db2f" candidate_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad4fc396f9d0488095791430ce88e06a" is_pre_election: true
I20260811 12:02:37.486255 12294 raft_consensus.cc:2454] T 26117350248d4571bbf0415710a6db2f P ad4fc396f9d0488095791430ce88e06a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bfe3a8d9f9d4ee787e8e0d39835f2e8 in term 0.
I20260811 12:02:37.486647 12534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26117350248d4571bbf0415710a6db2f" candidate_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "681003820cda4640af8791856d6f2992" is_pre_election: true
I20260811 12:02:37.486873 12644 leader_election.cc:302] T 26117350248d4571bbf0415710a6db2f P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bfe3a8d9f9d4ee787e8e0d39835f2e8, ad4fc396f9d0488095791430ce88e06a; no voters:
I20260811 12:02:37.487067 12534 raft_consensus.cc:2454] T 26117350248d4571bbf0415710a6db2f P 681003820cda4640af8791856d6f2992 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bfe3a8d9f9d4ee787e8e0d39835f2e8 in term 0.
I20260811 12:02:37.487274 12865 raft_consensus.cc:2781] T 26117350248d4571bbf0415710a6db2f P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:02:37.487797 12865 raft_consensus.cc:482] T 26117350248d4571bbf0415710a6db2f P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:02:37.487907 12865 raft_consensus.cc:3037] T 26117350248d4571bbf0415710a6db2f P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:02:37.490444 12865 raft_consensus.cc:504] T 26117350248d4571bbf0415710a6db2f P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad4fc396f9d0488095791430ce88e06a" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33027 } } peers { permanent_uuid: "681003820cda4640af8791856d6f2992" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43509 } } peers { permanent_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39597 } }
I20260811 12:02:37.491382 12865 leader_election.cc:288] T 26117350248d4571bbf0415710a6db2f P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [CANDIDATE]: Term 1 election: Requested vote from peers ad4fc396f9d0488095791430ce88e06a (127.9.221.129:33027), 681003820cda4640af8791856d6f2992 (127.9.221.130:43509)
I20260811 12:02:37.492087 12294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26117350248d4571bbf0415710a6db2f" candidate_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad4fc396f9d0488095791430ce88e06a"
I20260811 12:02:37.492250 12534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26117350248d4571bbf0415710a6db2f" candidate_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "681003820cda4640af8791856d6f2992"
I20260811 12:02:37.492421 12294 raft_consensus.cc:3037] T 26117350248d4571bbf0415710a6db2f P ad4fc396f9d0488095791430ce88e06a [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:02:37.492617 12534 raft_consensus.cc:3037] T 26117350248d4571bbf0415710a6db2f P 681003820cda4640af8791856d6f2992 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:02:37.496373 12534 raft_consensus.cc:2454] T 26117350248d4571bbf0415710a6db2f P 681003820cda4640af8791856d6f2992 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bfe3a8d9f9d4ee787e8e0d39835f2e8 in term 1.
I20260811 12:02:37.496407 12294 raft_consensus.cc:2454] T 26117350248d4571bbf0415710a6db2f P ad4fc396f9d0488095791430ce88e06a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bfe3a8d9f9d4ee787e8e0d39835f2e8 in term 1.
I20260811 12:02:37.496999 12644 leader_election.cc:302] T 26117350248d4571bbf0415710a6db2f P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bfe3a8d9f9d4ee787e8e0d39835f2e8, ad4fc396f9d0488095791430ce88e06a; no voters:
I20260811 12:02:37.497403 12865 raft_consensus.cc:2781] T 26117350248d4571bbf0415710a6db2f P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:02:37.497637 12865 raft_consensus.cc:686] T 26117350248d4571bbf0415710a6db2f P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [term 1 LEADER]: Becoming Leader. State: Replica: 3bfe3a8d9f9d4ee787e8e0d39835f2e8, State: Running, Role: LEADER
I20260811 12:02:37.498020 12865 consensus_queue.cc:237] T 26117350248d4571bbf0415710a6db2f P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad4fc396f9d0488095791430ce88e06a" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33027 } } peers { permanent_uuid: "681003820cda4640af8791856d6f2992" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43509 } } peers { permanent_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39597 } }
I20260811 12:02:37.502101 12015 catalog_manager.cc:5495] T 26117350248d4571bbf0415710a6db2f P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3bfe3a8d9f9d4ee787e8e0d39835f2e8 (127.9.221.131). New cstate: current_term: 1 leader_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad4fc396f9d0488095791430ce88e06a" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "681003820cda4640af8791856d6f2992" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39597 } health_report { overall_health: HEALTHY } } }
Aug 11 12:02:37 dist-test-slave-mjjr krb5kdc[11933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449754, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:02:37.604059 12920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:37.583002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:43507 (local address 127.0.0.1:51856)
0811 12:02:37.584508 (+ 1506us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:37.584524 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:37.584741 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:37.585077 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:37.585081 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:37.586424 (+ 1343us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:02:37.587021 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:37.587034 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:37.588429 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:37.588434 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:37.589740 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:37.589753 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:37.590705 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:37.590712 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:37.591014 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:37.591905 (+ 891us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:02:37.591923 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:02:37.595019 (+ 3096us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:02:37.600701 (+ 5682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:37.600715 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:37.600723 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:37.601320 (+ 597us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:37.601875 (+ 555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:37.601879 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:37.601881 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:37.602076 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:37.602967 (+ 891us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:02:37.602973 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:02:37.603131 (+ 158us) client_negotiation.cc:769] Sending connection context
0811 12:02:37.603340 (+ 209us) client_negotiation.cc:240] Negotiation successful
0811 12:02:37.603620 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1364,"thread_start_us":1098,"threads_started":1}
I20260811 12:02:37.604446 12918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:37.582631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51856 (local address 127.9.221.190:43507)
0811 12:02:37.584112 (+ 1481us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:37.584125 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:37.584623 (+ 498us) server_negotiation.cc:407] Connection header received
0811 12:02:37.584870 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:37.584875 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:37.584943 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:37.585056 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:37.587185 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:37.588262 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:37.589883 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:37.590505 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:37.590707 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:37.595324 (+ 4617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:37.595333 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:37.595339 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:37.595395 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:37.600445 (+ 5050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:37.601445 (+ 1000us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:37.601451 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:37.601454 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:37.601751 (+ 297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:37.602220 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:37.602233 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:37.602236 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:37.602784 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:37.602981 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:37.603483 (+ 502us) server_negotiation.cc:299] Negotiation successful
0811 12:02:37.603883 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":61,"server-negotiator.queue_time_us":1197,"thread_start_us":791,"threads_started":1}
I20260811 12:02:37.611202 12015 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:51856:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 12:02:37.612462 12015 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:02:37.621243 12249 tablet_service.cc:1467] Processing CreateTablet for tablet 23a2bdc0d1204f27a8e2bc22646b6dd4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=246c6461bbc34f948ae9ef9dfc69a5f1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 12:02:37.621528 12709 tablet_service.cc:1467] Processing CreateTablet for tablet 23a2bdc0d1204f27a8e2bc22646b6dd4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=246c6461bbc34f948ae9ef9dfc69a5f1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 12:02:37.621524 12500 tablet_service.cc:1467] Processing CreateTablet for tablet 23a2bdc0d1204f27a8e2bc22646b6dd4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=246c6461bbc34f948ae9ef9dfc69a5f1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 12:02:37.622081 12249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23a2bdc0d1204f27a8e2bc22646b6dd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:02:37.622295 12709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23a2bdc0d1204f27a8e2bc22646b6dd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:02:37.622408 12500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23a2bdc0d1204f27a8e2bc22646b6dd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:02:37.633721 12857 tablet_bootstrap.cc:492] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P ad4fc396f9d0488095791430ce88e06a: Bootstrap starting.
I20260811 12:02:37.636078 12855 tablet_bootstrap.cc:492] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8: Bootstrap starting.
I20260811 12:02:37.637059 12857 tablet_bootstrap.cc:654] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P ad4fc396f9d0488095791430ce88e06a: Neither blocks nor log segments found. Creating new log.
I20260811 12:02:37.637709 12861 tablet_bootstrap.cc:492] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P 681003820cda4640af8791856d6f2992: Bootstrap starting.
I20260811 12:02:37.639978 12861 tablet_bootstrap.cc:654] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P 681003820cda4640af8791856d6f2992: Neither blocks nor log segments found. Creating new log.
I20260811 12:02:37.641566 12855 tablet_bootstrap.cc:654] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8: Neither blocks nor log segments found. Creating new log.
I20260811 12:02:37.642971 12857 tablet_bootstrap.cc:492] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P ad4fc396f9d0488095791430ce88e06a: No bootstrap required, opened a new log
I20260811 12:02:37.643395 12857 ts_tablet_manager.cc:1397] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P ad4fc396f9d0488095791430ce88e06a: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.006s
I20260811 12:02:37.645273 12855 tablet_bootstrap.cc:492] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8: No bootstrap required, opened a new log
I20260811 12:02:37.645493 12855 ts_tablet_manager.cc:1397] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.001s
I20260811 12:02:37.646260 12857 raft_consensus.cc:348] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P ad4fc396f9d0488095791430ce88e06a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad4fc396f9d0488095791430ce88e06a" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33027 } } peers { permanent_uuid: "681003820cda4640af8791856d6f2992" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43509 } } peers { permanent_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39597 } }
I20260811 12:02:37.646747 12857 raft_consensus.cc:374] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P ad4fc396f9d0488095791430ce88e06a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:02:37.646977 12857 raft_consensus.cc:729] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P ad4fc396f9d0488095791430ce88e06a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad4fc396f9d0488095791430ce88e06a, State: Initialized, Role: FOLLOWER
I20260811 12:02:37.646864 12855 raft_consensus.cc:348] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad4fc396f9d0488095791430ce88e06a" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33027 } } peers { permanent_uuid: "681003820cda4640af8791856d6f2992" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43509 } } peers { permanent_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39597 } }
I20260811 12:02:37.647251 12855 raft_consensus.cc:374] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:02:37.647434 12855 raft_consensus.cc:729] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bfe3a8d9f9d4ee787e8e0d39835f2e8, State: Initialized, Role: FOLLOWER
I20260811 12:02:37.647630 12857 consensus_queue.cc:260] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P ad4fc396f9d0488095791430ce88e06a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad4fc396f9d0488095791430ce88e06a" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33027 } } peers { permanent_uuid: "681003820cda4640af8791856d6f2992" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43509 } } peers { permanent_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39597 } }
I20260811 12:02:37.648550 12862 tablet_replica.cc:439] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P ad4fc396f9d0488095791430ce88e06a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad4fc396f9d0488095791430ce88e06a" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33027 } } peers { permanent_uuid: "681003820cda4640af8791856d6f2992" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43509 } } peers { permanent_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39597 } } }
I20260811 12:02:37.649132 12862 tablet_replica.cc:442] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P ad4fc396f9d0488095791430ce88e06a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:02:37.651211 12857 ts_tablet_manager.cc:1428] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P ad4fc396f9d0488095791430ce88e06a: Time spent starting tablet: real 0.008s user 0.002s sys 0.001s
I20260811 12:02:37.647899 12855 consensus_queue.cc:260] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad4fc396f9d0488095791430ce88e06a" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33027 } } peers { permanent_uuid: "681003820cda4640af8791856d6f2992" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43509 } } peers { permanent_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39597 } }
I20260811 12:02:37.652586 12865 tablet_replica.cc:439] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad4fc396f9d0488095791430ce88e06a" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33027 } } peers { permanent_uuid: "681003820cda4640af8791856d6f2992" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43509 } } peers { permanent_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39597 } } }
I20260811 12:02:37.653349 12865 tablet_replica.cc:442] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:02:37.653716 12861 tablet_bootstrap.cc:492] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P 681003820cda4640af8791856d6f2992: No bootstrap required, opened a new log
I20260811 12:02:37.654003 12861 ts_tablet_manager.cc:1397] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P 681003820cda4640af8791856d6f2992: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.013s
I20260811 12:02:37.655210 12861 raft_consensus.cc:348] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P 681003820cda4640af8791856d6f2992 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad4fc396f9d0488095791430ce88e06a" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33027 } } peers { permanent_uuid: "681003820cda4640af8791856d6f2992" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43509 } } peers { permanent_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39597 } }
I20260811 12:02:37.655874 12861 raft_consensus.cc:374] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P 681003820cda4640af8791856d6f2992 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:02:37.656018 12861 raft_consensus.cc:729] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P 681003820cda4640af8791856d6f2992 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 681003820cda4640af8791856d6f2992, State: Initialized, Role: FOLLOWER
I20260811 12:02:37.656404 12861 consensus_queue.cc:260] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P 681003820cda4640af8791856d6f2992 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad4fc396f9d0488095791430ce88e06a" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33027 } } peers { permanent_uuid: "681003820cda4640af8791856d6f2992" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43509 } } peers { permanent_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39597 } }
I20260811 12:02:37.657145 12867 tablet_replica.cc:439] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P 681003820cda4640af8791856d6f2992: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad4fc396f9d0488095791430ce88e06a" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33027 } } peers { permanent_uuid: "681003820cda4640af8791856d6f2992" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43509 } } peers { permanent_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39597 } } }
I20260811 12:02:37.657567 12867 tablet_replica.cc:442] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P 681003820cda4640af8791856d6f2992: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:02:37.658020 12855 ts_tablet_manager.cc:1428] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8: Time spent starting tablet: real 0.011s user 0.004s sys 0.000s
I20260811 12:02:37.658080 12861 ts_tablet_manager.cc:1428] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P 681003820cda4640af8791856d6f2992: Time spent starting tablet: real 0.004s user 0.002s sys 0.002s
I20260811 12:02:37.827056 12862 raft_consensus.cc:482] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P ad4fc396f9d0488095791430ce88e06a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:02:37.827337 12862 raft_consensus.cc:504] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P ad4fc396f9d0488095791430ce88e06a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad4fc396f9d0488095791430ce88e06a" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33027 } } peers { permanent_uuid: "681003820cda4640af8791856d6f2992" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43509 } } peers { permanent_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39597 } }
I20260811 12:02:37.829082 12862 leader_election.cc:288] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P ad4fc396f9d0488095791430ce88e06a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 681003820cda4640af8791856d6f2992 (127.9.221.130:43509), 3bfe3a8d9f9d4ee787e8e0d39835f2e8 (127.9.221.131:39597)
I20260811 12:02:37.838444 12931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:37.829383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:41915 (local address 127.9.221.130:43509)
0811 12:02:37.830260 (+ 877us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:37.830268 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:37.830289 (+ 21us) server_negotiation.cc:407] Connection header received
0811 12:02:37.830404 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:37.830412 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:37.830530 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:37.831081 (+ 551us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:02:37.831939 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:37.832917 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:37.835663 (+ 2746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:37.837296 (+ 1633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:37.837494 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:37.837577 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:37.837969 (+ 392us) server_negotiation.cc:299] Negotiation successful
0811 12:02:37.838070 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":755,"thread_start_us":638,"threads_started":1}
I20260811 12:02:37.838956 12927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:37.829245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.130:43509 (local address 127.9.221.129:41915)
0811 12:02:37.829687 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:37.829704 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:37.829819 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:37.830740 (+ 921us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:37.830748 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:37.830788 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:02:37.831535 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:37.831546 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:37.833112 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:37.833118 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:37.835435 (+ 2317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:37.835447 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:37.837492 (+ 2045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:37.837499 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:37.837786 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:37.837834 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 12:02:37.837936 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 12:02:37.838065 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"mutex_wait_us":19,"thread_start_us":249,"threads_started":1}
I20260811 12:02:37.840112 12534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23a2bdc0d1204f27a8e2bc22646b6dd4" candidate_uuid: "ad4fc396f9d0488095791430ce88e06a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "681003820cda4640af8791856d6f2992" is_pre_election: true
I20260811 12:02:37.840492 12534 raft_consensus.cc:2454] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P 681003820cda4640af8791856d6f2992 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad4fc396f9d0488095791430ce88e06a in term 0.
I20260811 12:02:37.840982 12934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:37.830148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:58235 (local address 127.9.221.131:39597)
0811 12:02:37.831376 (+ 1228us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:37.831382 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:37.831402 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:02:37.831540 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:37.831544 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:37.831699 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:37.831815 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:02:37.832815 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:37.834490 (+ 1675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:37.837577 (+ 3087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:37.839385 (+ 1808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:37.839727 (+ 342us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:37.839805 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:37.840272 (+ 467us) server_negotiation.cc:299] Negotiation successful
0811 12:02:37.840472 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1069,"thread_start_us":878,"threads_started":1}
I20260811 12:02:37.841142 12933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:37.830009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.131:39597 (local address 127.9.221.129:58235)
0811 12:02:37.831222 (+ 1213us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:37.831242 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:37.831374 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:37.831827 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:37.831831 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:37.831854 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:02:37.832563 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:37.832573 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:37.834611 (+ 2038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:37.834615 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:37.837386 (+ 2771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:37.837405 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:37.839810 (+ 2405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:37.839817 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:37.840072 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:37.840127 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 12:02:37.840226 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 12:02:37.840362 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":745,"mutex_wait_us":1,"thread_start_us":484,"threads_started":1}
I20260811 12:02:37.841265 12162 leader_election.cc:302] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P ad4fc396f9d0488095791430ce88e06a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 681003820cda4640af8791856d6f2992, ad4fc396f9d0488095791430ce88e06a; no voters:
I20260811 12:02:37.841691 12862 raft_consensus.cc:2781] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P ad4fc396f9d0488095791430ce88e06a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:02:37.841945 12862 raft_consensus.cc:482] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P ad4fc396f9d0488095791430ce88e06a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:02:37.842165 12862 raft_consensus.cc:3037] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P ad4fc396f9d0488095791430ce88e06a [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:02:37.842736 12744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23a2bdc0d1204f27a8e2bc22646b6dd4" candidate_uuid: "ad4fc396f9d0488095791430ce88e06a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" is_pre_election: true
I20260811 12:02:37.843324 12744 raft_consensus.cc:2454] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad4fc396f9d0488095791430ce88e06a in term 0.
I20260811 12:02:37.845263 12862 raft_consensus.cc:504] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P ad4fc396f9d0488095791430ce88e06a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad4fc396f9d0488095791430ce88e06a" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33027 } } peers { permanent_uuid: "681003820cda4640af8791856d6f2992" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43509 } } peers { permanent_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39597 } }
I20260811 12:02:37.846030 12862 leader_election.cc:288] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P ad4fc396f9d0488095791430ce88e06a [CANDIDATE]: Term 1 election: Requested vote from peers 681003820cda4640af8791856d6f2992 (127.9.221.130:43509), 3bfe3a8d9f9d4ee787e8e0d39835f2e8 (127.9.221.131:39597)
I20260811 12:02:37.846477 12534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23a2bdc0d1204f27a8e2bc22646b6dd4" candidate_uuid: "ad4fc396f9d0488095791430ce88e06a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "681003820cda4640af8791856d6f2992"
I20260811 12:02:37.846539 12744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23a2bdc0d1204f27a8e2bc22646b6dd4" candidate_uuid: "ad4fc396f9d0488095791430ce88e06a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8"
I20260811 12:02:37.846755 12744 raft_consensus.cc:3037] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:02:37.846771 12534 raft_consensus.cc:3037] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P 681003820cda4640af8791856d6f2992 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:02:37.849296 12534 raft_consensus.cc:2454] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P 681003820cda4640af8791856d6f2992 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad4fc396f9d0488095791430ce88e06a in term 1.
I20260811 12:02:37.849658 12744 raft_consensus.cc:2454] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad4fc396f9d0488095791430ce88e06a in term 1.
I20260811 12:02:37.849725 12162 leader_election.cc:302] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P ad4fc396f9d0488095791430ce88e06a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 681003820cda4640af8791856d6f2992, ad4fc396f9d0488095791430ce88e06a; no voters:
I20260811 12:02:37.850118 12862 raft_consensus.cc:2781] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P ad4fc396f9d0488095791430ce88e06a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:02:37.850955 12862 raft_consensus.cc:686] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P ad4fc396f9d0488095791430ce88e06a [term 1 LEADER]: Becoming Leader. State: Replica: ad4fc396f9d0488095791430ce88e06a, State: Running, Role: LEADER
I20260811 12:02:37.851450 12862 consensus_queue.cc:237] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P ad4fc396f9d0488095791430ce88e06a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad4fc396f9d0488095791430ce88e06a" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33027 } } peers { permanent_uuid: "681003820cda4640af8791856d6f2992" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43509 } } peers { permanent_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39597 } }
I20260811 12:02:37.854403 12936 tablet_replica.cc:439] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P ad4fc396f9d0488095791430ce88e06a: TxnStatusTablet state changed. Reason: New leader ad4fc396f9d0488095791430ce88e06a. Latest consensus state: current_term: 1 leader_uuid: "ad4fc396f9d0488095791430ce88e06a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad4fc396f9d0488095791430ce88e06a" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33027 } } peers { permanent_uuid: "681003820cda4640af8791856d6f2992" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43509 } } peers { permanent_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39597 } } }
I20260811 12:02:37.855273 12936 tablet_replica.cc:442] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P ad4fc396f9d0488095791430ce88e06a: This TxnStatusTablet replica's current role is: LEADER
I20260811 12:02:37.856729 12015 catalog_manager.cc:5495] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P ad4fc396f9d0488095791430ce88e06a reported cstate change: term changed from 0 to 1, leader changed from <none> to ad4fc396f9d0488095791430ce88e06a (127.9.221.129). New cstate: current_term: 1 leader_uuid: "ad4fc396f9d0488095791430ce88e06a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad4fc396f9d0488095791430ce88e06a" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33027 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "681003820cda4640af8791856d6f2992" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39597 } health_report { overall_health: UNKNOWN } } }
I20260811 12:02:37.857709 12943 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 12:02:37.858052 12943 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 12:02:37.890193 12920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:37.881044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:33027 (local address 127.0.0.1:60730)
0811 12:02:37.881618 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:37.881641 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:37.881836 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:37.882401 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:37.882407 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:37.882421 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 12:02:37.882795 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:37.882809 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:37.884516 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:37.884522 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:37.886839 (+ 2317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:37.886854 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:37.887868 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:37.887886 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:37.888197 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:37.888288 (+ 91us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 12:02:37.889421 (+ 1133us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 12:02:37.889444 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 12:02:37.889550 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 12:02:37.889754 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371}
I20260811 12:02:37.890317 12946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:37.880759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60730 (local address 127.9.221.129:33027)
0811 12:02:37.881722 (+ 963us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:37.881726 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:37.881750 (+ 24us) server_negotiation.cc:407] Connection header received
0811 12:02:37.882124 (+ 374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:37.882128 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:37.882206 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:37.882340 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:02:37.883016 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:37.884295 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:37.886987 (+ 2692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:37.887626 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:37.887786 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:37.888571 (+ 785us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:02:37.889284 (+ 713us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:02:37.889394 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:37.889576 (+ 182us) server_negotiation.cc:299] Negotiation successful
0811 12:02:37.889647 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":847,"thread_start_us":675,"threads_started":1}
I20260811 12:02:37.897150 12534 raft_consensus.cc:1261] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P 681003820cda4640af8791856d6f2992 [term 1 FOLLOWER]: Refusing update from remote peer ad4fc396f9d0488095791430ce88e06a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:02:37.897535 12744 raft_consensus.cc:1261] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [term 1 FOLLOWER]: Refusing update from remote peer ad4fc396f9d0488095791430ce88e06a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:02:37.898219 12936 consensus_queue.cc:1035] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P ad4fc396f9d0488095791430ce88e06a [LEADER]: Connected to new peer: Peer: permanent_uuid: "681003820cda4640af8791856d6f2992" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:02:37.898664 12862 consensus_queue.cc:1035] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P ad4fc396f9d0488095791430ce88e06a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:02:37.907960 12865 tablet_replica.cc:439] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8: TxnStatusTablet state changed. Reason: New leader ad4fc396f9d0488095791430ce88e06a. Latest consensus state: current_term: 1 leader_uuid: "ad4fc396f9d0488095791430ce88e06a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad4fc396f9d0488095791430ce88e06a" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33027 } } peers { permanent_uuid: "681003820cda4640af8791856d6f2992" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43509 } } peers { permanent_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39597 } } }
I20260811 12:02:37.908520 12865 tablet_replica.cc:442] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:02:37.913012 12867 tablet_replica.cc:439] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P 681003820cda4640af8791856d6f2992: TxnStatusTablet state changed. Reason: New leader ad4fc396f9d0488095791430ce88e06a. Latest consensus state: current_term: 1 leader_uuid: "ad4fc396f9d0488095791430ce88e06a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad4fc396f9d0488095791430ce88e06a" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33027 } } peers { permanent_uuid: "681003820cda4640af8791856d6f2992" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43509 } } peers { permanent_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39597 } } }
I20260811 12:02:37.913493 12867 tablet_replica.cc:442] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P 681003820cda4640af8791856d6f2992: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:02:37.915725 12862 tablet_replica.cc:439] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P ad4fc396f9d0488095791430ce88e06a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ad4fc396f9d0488095791430ce88e06a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad4fc396f9d0488095791430ce88e06a" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33027 } } peers { permanent_uuid: "681003820cda4640af8791856d6f2992" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43509 } } peers { permanent_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39597 } } }
I20260811 12:02:37.916239 12862 tablet_replica.cc:442] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P ad4fc396f9d0488095791430ce88e06a: This TxnStatusTablet replica's current role is: LEADER
I20260811 12:02:37.918803 12948 mvcc.cc:205] Tried to move back new op lower bound from 7317298208338087936 to 7317298208163713024. Current Snapshot: MvccSnapshot[applied={T|T < 7317298208338087936}]
I20260811 12:02:37.922911 12862 tablet_replica.cc:439] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P ad4fc396f9d0488095791430ce88e06a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ad4fc396f9d0488095791430ce88e06a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad4fc396f9d0488095791430ce88e06a" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33027 } } peers { permanent_uuid: "681003820cda4640af8791856d6f2992" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43509 } } peers { permanent_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39597 } } }
I20260811 12:02:37.923317 12862 tablet_replica.cc:442] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P ad4fc396f9d0488095791430ce88e06a: This TxnStatusTablet replica's current role is: LEADER
I20260811 12:02:37.925674 12865 tablet_replica.cc:439] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ad4fc396f9d0488095791430ce88e06a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad4fc396f9d0488095791430ce88e06a" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33027 } } peers { permanent_uuid: "681003820cda4640af8791856d6f2992" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43509 } } peers { permanent_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39597 } } }
I20260811 12:02:37.925972 12865 tablet_replica.cc:442] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P 3bfe3a8d9f9d4ee787e8e0d39835f2e8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:02:37.931610 12867 tablet_replica.cc:439] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P 681003820cda4640af8791856d6f2992: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ad4fc396f9d0488095791430ce88e06a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad4fc396f9d0488095791430ce88e06a" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33027 } } peers { permanent_uuid: "681003820cda4640af8791856d6f2992" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43509 } } peers { permanent_uuid: "3bfe3a8d9f9d4ee787e8e0d39835f2e8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39597 } } }
I20260811 12:02:37.931981 12867 tablet_replica.cc:442] T 23a2bdc0d1204f27a8e2bc22646b6dd4 P 681003820cda4640af8791856d6f2992: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:02:37.948529 12946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:37.941937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:36197 (local address 127.9.221.129:33027)
0811 12:02:37.942106 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:37.942110 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:37.943045 (+ 935us) server_negotiation.cc:407] Connection header received
0811 12:02:37.943307 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:37.943313 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:37.943393 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:37.943499 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:02:37.944171 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:37.944997 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:37.946509 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:37.947029 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:37.947187 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:37.947710 (+ 523us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:02:37.947817 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:02:37.947921 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:37.948071 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 12:02:37.948151 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 12:02:37.948846 12966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:37.941906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:33027 (local address 127.9.221.131:36197)
0811 12:02:37.942925 (+ 1019us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:37.942941 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:37.943159 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:37.943531 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:37.943535 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:37.943548 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 12:02:37.944014 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:37.944024 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:37.945158 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:37.945165 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:37.946338 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:37.946350 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:37.947245 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:37.947253 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:37.947427 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:37.947488 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 12:02:37.947924 (+ 436us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 12:02:37.947939 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 12:02:37.948043 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 12:02:37.948175 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":843,"thread_start_us":657,"threads_started":1}
I20260811 12:02:37.959406 12534 raft_consensus.cc:1261] T 26117350248d4571bbf0415710a6db2f P 681003820cda4640af8791856d6f2992 [term 1 FOLLOWER]: Refusing update from remote peer 3bfe3a8d9f9d4ee787e8e0d39835f2e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:02:37.960023 12294 raft_consensus.cc:1261] T 26117350248d4571bbf0415710a6db2f P ad4fc396f9d0488095791430ce88e06a [term 1 FOLLOWER]: Refusing update from remote peer 3bfe3a8d9f9d4ee787e8e0d39835f2e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:02:37.960101 12865 consensus_queue.cc:1035] T 26117350248d4571bbf0415710a6db2f P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "681003820cda4640af8791856d6f2992" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 43509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:02:37.960786 12865 consensus_queue.cc:1035] T 26117350248d4571bbf0415710a6db2f P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad4fc396f9d0488095791430ce88e06a" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:02:38.015681 12934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:38.005526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:33637 (local address 127.9.221.131:39597)
0811 12:02:38.005706 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:38.005710 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:38.007831 (+ 2121us) server_negotiation.cc:407] Connection header received
0811 12:02:38.008171 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:38.008176 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:38.008250 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:38.008350 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:02:38.009346 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:38.010336 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:38.013014 (+ 2678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:38.013687 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:38.013806 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:38.014650 (+ 844us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:02:38.014743 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:02:38.014850 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:38.015109 (+ 259us) server_negotiation.cc:299] Negotiation successful
0811 12:02:38.015190 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 12:02:38.016328 12978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:38.005752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.131:39597 (local address 127.9.221.129:33637)
0811 12:02:38.007723 (+ 1971us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:38.007744 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:38.008027 (+ 283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:38.008597 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:38.008602 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:38.008614 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 12:02:38.009183 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:38.009194 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:38.010475 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:38.010480 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:38.012869 (+ 2389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:38.012882 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:38.014035 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:38.014043 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:38.014361 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:38.014451 (+ 90us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 12:02:38.014930 (+ 479us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 12:02:38.014944 (+ 14us) client_negotiation.cc:769] Sending connection context
0811 12:02:38.015047 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 12:02:38.015206 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1706,"thread_start_us":1929,"threads_started":1}
I20260811 12:02:38.052534 12012 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51854:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:02:38.052778 12012 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51854:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:02:38.056202 12012 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 186edf6d1f844a7c942fc1382fa1c773: Sending DeleteTablet for 3 replicas of tablet 26117350248d4571bbf0415710a6db2f
I20260811 12:02:38.057199 12249 tablet_service.cc:1514] Processing DeleteTablet for tablet 26117350248d4571bbf0415710a6db2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:02:38 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:60714
I20260811 12:02:38.057247 12709 tablet_service.cc:1514] Processing DeleteTablet for tablet 26117350248d4571bbf0415710a6db2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:02:38 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:35782
I20260811 12:02:38.059453 12500 tablet_service.cc:1514] Processing DeleteTablet for tablet 26117350248d4571bbf0415710a6db2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:02:38 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:38794
I20260811 12:02:38.072652 12986 tablet_replica.cc:330] T 26117350248d4571bbf0415710a6db2f P 681003820cda4640af8791856d6f2992: stopping tablet replica
I20260811 12:02:38.073129 12986 raft_consensus.cc:2229] T 26117350248d4571bbf0415710a6db2f P 681003820cda4640af8791856d6f2992 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:02:38.073796 12986 raft_consensus.cc:2258] T 26117350248d4571bbf0415710a6db2f P 681003820cda4640af8791856d6f2992 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:02:38.075338 12986 ts_tablet_manager.cc:1905] T 26117350248d4571bbf0415710a6db2f P 681003820cda4640af8791856d6f2992: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:02:38.082415 12984 tablet_replica.cc:330] T 26117350248d4571bbf0415710a6db2f P 3bfe3a8d9f9d4ee787e8e0d39835f2e8: stopping tablet replica
I20260811 12:02:38.082753 12984 raft_consensus.cc:2229] T 26117350248d4571bbf0415710a6db2f P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:02:38.083276 12984 raft_consensus.cc:2258] T 26117350248d4571bbf0415710a6db2f P 3bfe3a8d9f9d4ee787e8e0d39835f2e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:02:38.084002 12981 tablet_replica.cc:330] T 26117350248d4571bbf0415710a6db2f P ad4fc396f9d0488095791430ce88e06a: stopping tablet replica
I20260811 12:02:38.084293 12981 raft_consensus.cc:2229] T 26117350248d4571bbf0415710a6db2f P ad4fc396f9d0488095791430ce88e06a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:02:38.084954 12984 ts_tablet_manager.cc:1905] T 26117350248d4571bbf0415710a6db2f P 3bfe3a8d9f9d4ee787e8e0d39835f2e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:02:38.084959 12981 raft_consensus.cc:2258] T 26117350248d4571bbf0415710a6db2f P ad4fc396f9d0488095791430ce88e06a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:02:38.090095 12986 ts_tablet_manager.cc:1918] T 26117350248d4571bbf0415710a6db2f P 681003820cda4640af8791856d6f2992: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:02:38.090410 12986 log.cc:1196] T 26117350248d4571bbf0415710a6db2f P 681003820cda4640af8791856d6f2992: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-1/wal/wals/26117350248d4571bbf0415710a6db2f
I20260811 12:02:38.091089 12986 ts_tablet_manager.cc:1939] T 26117350248d4571bbf0415710a6db2f P 681003820cda4640af8791856d6f2992: Deleting consensus metadata
I20260811 12:02:38.092552 11980 catalog_manager.cc:4841] TS 681003820cda4640af8791856d6f2992 (127.9.221.130:43509): tablet 26117350248d4571bbf0415710a6db2f (table test-table [id=93f5274b97884b4783bf4329357f1cb1]) successfully deleted
I20260811 12:02:38.093796 12981 ts_tablet_manager.cc:1905] T 26117350248d4571bbf0415710a6db2f P ad4fc396f9d0488095791430ce88e06a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:02:38.094466 12984 ts_tablet_manager.cc:1918] T 26117350248d4571bbf0415710a6db2f P 3bfe3a8d9f9d4ee787e8e0d39835f2e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:02:38.094765 12984 log.cc:1196] T 26117350248d4571bbf0415710a6db2f P 3bfe3a8d9f9d4ee787e8e0d39835f2e8: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-2/wal/wals/26117350248d4571bbf0415710a6db2f
I20260811 12:02:38.095853 12984 ts_tablet_manager.cc:1939] T 26117350248d4571bbf0415710a6db2f P 3bfe3a8d9f9d4ee787e8e0d39835f2e8: Deleting consensus metadata
I20260811 12:02:38.097100 11980 catalog_manager.cc:4841] TS 3bfe3a8d9f9d4ee787e8e0d39835f2e8 (127.9.221.131:39597): tablet 26117350248d4571bbf0415710a6db2f (table test-table [id=93f5274b97884b4783bf4329357f1cb1]) successfully deleted
I20260811 12:02:38.103492 12981 ts_tablet_manager.cc:1918] T 26117350248d4571bbf0415710a6db2f P ad4fc396f9d0488095791430ce88e06a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:02:38.103853 12981 log.cc:1196] T 26117350248d4571bbf0415710a6db2f P ad4fc396f9d0488095791430ce88e06a: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449743084027-10102-0/minicluster-data/ts-0/wal/wals/26117350248d4571bbf0415710a6db2f
I20260811 12:02:38.104612 12981 ts_tablet_manager.cc:1939] T 26117350248d4571bbf0415710a6db2f P ad4fc396f9d0488095791430ce88e06a: Deleting consensus metadata
Aug 11 12:02:38 dist-test-slave-mjjr krb5kdc[11933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449756, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.221.129@KRBTEST.COM
I20260811 12:02:38.105782 11985 catalog_manager.cc:4841] TS ad4fc396f9d0488095791430ce88e06a (127.9.221.129:33027): tablet 26117350248d4571bbf0415710a6db2f (table test-table [id=93f5274b97884b4783bf4329357f1cb1]) successfully deleted
I20260811 12:02:38.117024 12946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:38.066456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60746 (local address 127.9.221.129:33027)
0811 12:02:38.066678 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:38.066686 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:38.074511 (+ 7825us) server_negotiation.cc:407] Connection header received
0811 12:02:38.075274 (+ 763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:38.075282 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:38.075421 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:38.075579 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:38.088049 (+ 12470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:38.089476 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:38.097139 (+ 7663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:38.098076 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:38.098217 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:38.107094 (+ 8877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:38.107125 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:38.107145 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:38.107186 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:38.112893 (+ 5707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:38.114005 (+ 1112us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:38.114014 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:38.114018 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:38.114212 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:38.114714 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:38.114722 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:38.114727 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:38.115343 (+ 616us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:38.115857 (+ 514us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:38.116119 (+ 262us) server_negotiation.cc:299] Negotiation successful
0811 12:02:38.116409 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260811 12:02:38.118027 12364 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60746
I20260811 12:02:38.143846 12990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:38.121724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51862 (local address 127.9.221.190:43507)
0811 12:02:38.124440 (+ 2716us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:38.124447 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:38.124466 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:02:38.124523 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:38.124528 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:38.124600 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:38.124712 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:38.126510 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:38.128400 (+ 1890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:38.129992 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:38.131032 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:38.131203 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:38.134666 (+ 3463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:38.134673 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:38.134678 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:38.134719 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:38.140114 (+ 5395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:38.141144 (+ 1030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:38.141151 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:38.141154 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:38.141450 (+ 296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:38.141875 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:38.141882 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:38.141885 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:38.142372 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:38.142712 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:38.142910 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 12:02:38.143222 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2567,"thread_start_us":2267,"threads_started":1}
W20260811 12:02:38.144582 12012 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51862
I20260811 12:02:38.148533 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 12132
I20260811 12:02:38.164139 12946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:38.149386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60760 (local address 127.9.221.129:33027)
0811 12:02:38.149625 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:38.149632 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:38.149851 (+ 219us) server_negotiation.cc:407] Connection header received
0811 12:02:38.149945 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:38.149951 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:38.150050 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:38.150178 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:38.151706 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:38.152499 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:38.153879 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:38.154368 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:38.154515 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:38.157632 (+ 3117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:38.157638 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:38.157643 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:38.157684 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:38.161451 (+ 3767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:38.162067 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:38.162072 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:38.162075 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:38.162228 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:38.162565 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:38.162570 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:38.162574 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:38.162936 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:38.163228 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:38.163432 (+ 204us) server_negotiation.cc:299] Negotiation successful
0811 12:02:38.163677 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260811 12:02:38.164872 12364 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60760
W20260811 12:02:38.165563 10102 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 12132: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:02:38.165858 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 12132
I20260811 12:02:38.193840 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 12375
Aug 11 12:02:38 dist-test-slave-mjjr krb5kdc[11933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449756, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.221.130@KRBTEST.COM
I20260811 12:02:38.210134 12931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:38.194587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38810 (local address 127.9.221.130:43509)
0811 12:02:38.194856 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:38.194862 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:38.194985 (+ 123us) server_negotiation.cc:407] Connection header received
0811 12:02:38.195188 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:38.195195 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:38.195268 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:38.195375 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:38.196897 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:38.197722 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:38.199304 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:38.199877 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:38.200022 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:38.204390 (+ 4368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:38.204407 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:38.204422 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:38.204462 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:38.207195 (+ 2733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:38.208002 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:38.208010 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:38.208013 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:38.208149 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:38.208555 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:38.208559 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:38.208561 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:38.208939 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:38.209283 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:38.209553 (+ 270us) server_negotiation.cc:299] Negotiation successful
0811 12:02:38.209788 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142}
W20260811 12:02:38.210850 12600 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38810
W20260811 12:02:38.211484 10102 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 12375: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:02:38.211724 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 12375
I20260811 12:02:38.234001 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 12613
Aug 11 12:02:38 dist-test-slave-mjjr krb5kdc[11933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449756, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.221.131@KRBTEST.COM
I20260811 12:02:38.249720 12934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:38.234802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35802 (local address 127.9.221.131:39597)
0811 12:02:38.235002 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:38.235007 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:38.235110 (+ 103us) server_negotiation.cc:407] Connection header received
0811 12:02:38.235356 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:38.235360 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:38.235439 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:38.235559 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:38.236935 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:38.237654 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:38.238835 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:38.239239 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:38.239328 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:38.243721 (+ 4393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:38.243738 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:38.243754 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:38.243793 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:38.246731 (+ 2938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:38.247435 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:38.247442 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:38.247445 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:38.247588 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:38.248049 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:38.248055 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:38.248058 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:38.248460 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:38.248761 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:38.249097 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 12:02:38.249390 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260811 12:02:38.250391 12818 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35802
W20260811 12:02:38.251034 10102 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 12613: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:02:38.251204 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 12613
I20260811 12:02:38.276677 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 11954
I20260811 12:02:38.277845 12076 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:02:38.394236 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 11954
2026-08-11T12:02:38Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5290 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:02:38 dist-test-slave-mjjr krb5kdc[12997](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:02:38 dist-test-slave-mjjr krb5kdc[12997](info): set up 2 sockets
Aug 11 12:02:38 dist-test-slave-mjjr krb5kdc[12997](info): commencing operation
krb5kdc: starting...
W20260811 12:02:39.697309 10102 mini_kdc.cc:121] Time spent starting KDC: real 1.250s user 0.000s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Aug 11 12:02:39 dist-test-slave-mjjr krb5kdc[12997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:02:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:02:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.221.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
I20260811 12:02:39.888124 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:41979
--webserver_interface=127.9.221.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:45699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:41979
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:02:40.211854 13022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:02:40.212149 13022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:02:40.212330 13022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:02:40.219521 13022 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:02:40.219666 13022 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 12:02:40.219770 13022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:02:40.219846 13022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:02:40.219906 13022 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:02:40.219965 13022 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:02:40.230141 13022 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:45699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:41979
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:41979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.221.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:02:40.231405 13022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:02:40.232733 13022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:02:40.244266 13031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:40.244328 13034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:40.244329 13030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:02:40.245838 13022 server_base.cc:1034] running on GCE node
I20260811 12:02:40.246542 13022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:02:40.247856 13022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:02:40.249075 13022 hybrid_clock.cc:648] HybridClock initialized: now 1786449760249017 us; error 34 us; skew 500 ppm
I20260811 12:02:40.249413 13022 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:02:40 dist-test-slave-mjjr krb5kdc[12997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449760, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:02:40.255784 13022 init.cc:377] Logged in from keytab as kudu/127.9.221.190@KRBTEST.COM (short username kudu)
I20260811 12:02:40.257195 13022 webserver.cc:460] Webserver started at http://127.9.221.190:35105/ using document root <none> and password file <none>
I20260811 12:02:40.257928 13022 fs_manager.cc:359] Metadata directory not provided
I20260811 12:02:40.258055 13022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:02:40.258384 13022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:02:40.261409 13022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "d924d872636d40318a5089eb6fccbf6c"
format_stamp: "Formatted at 2026-08-11 12:02:40 on dist-test-slave-mjjr"
server_key: "d5e45086f1b9812de35b5530090ff52a"
server_key_iv: "94caac6b7d05c1b019d6f5a92b671205"
server_key_version: "encryptionkey@0"
I20260811 12:02:40.262189 13022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "d924d872636d40318a5089eb6fccbf6c"
format_stamp: "Formatted at 2026-08-11 12:02:40 on dist-test-slave-mjjr"
server_key: "d5e45086f1b9812de35b5530090ff52a"
server_key_iv: "94caac6b7d05c1b019d6f5a92b671205"
server_key_version: "encryptionkey@0"
I20260811 12:02:40.267220 13022 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 12:02:40.270551 13046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:40.271908 13022 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:02:40.272107 13022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "d924d872636d40318a5089eb6fccbf6c"
format_stamp: "Formatted at 2026-08-11 12:02:40 on dist-test-slave-mjjr"
server_key: "d5e45086f1b9812de35b5530090ff52a"
server_key_iv: "94caac6b7d05c1b019d6f5a92b671205"
server_key_version: "encryptionkey@0"
I20260811 12:02:40.272347 13022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:02:40.289517 13022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:02:40.293363 13022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:02:40.293635 13022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:02:40.315587 13156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.190:41979 every 8 connection(s)
I20260811 12:02:40.315585 13022 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.190:41979
I20260811 12:02:40.316962 13022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:02:40.320712 13159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:02:40.326810 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 13022
I20260811 12:02:40.327028 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:02:40.327925 10102 external_mini_cluster.cc:1428] Setting key ffce7aacdb93ab07c9717f1a2325df00
I20260811 12:02:40.331784 13159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d924d872636d40318a5089eb6fccbf6c: Bootstrap starting.
I20260811 12:02:40.335253 13159 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d924d872636d40318a5089eb6fccbf6c: Neither blocks nor log segments found. Creating new log.
I20260811 12:02:40.336647 13159 log.cc:824] T 00000000000000000000000000000000 P d924d872636d40318a5089eb6fccbf6c: Log is configured to *not* fsync() on all Append() calls
I20260811 12:02:40.340436 13159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d924d872636d40318a5089eb6fccbf6c: No bootstrap required, opened a new log
Aug 11 12:02:40 dist-test-slave-mjjr krb5kdc[12997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:02:40.349267 13159 raft_consensus.cc:348] T 00000000000000000000000000000000 P d924d872636d40318a5089eb6fccbf6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d924d872636d40318a5089eb6fccbf6c" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 41979 } }
I20260811 12:02:40.349613 13159 raft_consensus.cc:374] T 00000000000000000000000000000000 P d924d872636d40318a5089eb6fccbf6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:02:40.349741 13159 raft_consensus.cc:729] T 00000000000000000000000000000000 P d924d872636d40318a5089eb6fccbf6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d924d872636d40318a5089eb6fccbf6c, State: Initialized, Role: FOLLOWER
I20260811 12:02:40.350330 13159 consensus_queue.cc:260] T 00000000000000000000000000000000 P d924d872636d40318a5089eb6fccbf6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d924d872636d40318a5089eb6fccbf6c" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 41979 } }
I20260811 12:02:40.350559 13159 raft_consensus.cc:388] T 00000000000000000000000000000000 P d924d872636d40318a5089eb6fccbf6c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:02:40.350709 13159 raft_consensus.cc:482] T 00000000000000000000000000000000 P d924d872636d40318a5089eb6fccbf6c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:02:40.350900 13159 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d924d872636d40318a5089eb6fccbf6c [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:02:40.352849 13163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:40.330240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39408 (local address 127.9.221.190:41979)
0811 12:02:40.331083 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:40.331094 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:40.331136 (+ 42us) server_negotiation.cc:407] Connection header received
0811 12:02:40.333008 (+ 1872us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:40.333029 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:40.333406 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:40.333718 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:40.335558 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:40.337534 (+ 1976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:40.339436 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:40.340141 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:40.340273 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:40.345022 (+ 4749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:40.345030 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:40.345047 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:40.345086 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:40.348358 (+ 3272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:40.349039 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:40.349051 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:40.349056 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:40.349225 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:40.349698 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:40.349704 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:40.349709 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:40.350424 (+ 715us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:40.350756 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:40.351311 (+ 555us) server_negotiation.cc:299] Negotiation successful
0811 12:02:40.351802 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":419,"threads_started":1}
I20260811 12:02:40.353497 13159 raft_consensus.cc:504] T 00000000000000000000000000000000 P d924d872636d40318a5089eb6fccbf6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d924d872636d40318a5089eb6fccbf6c" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 41979 } }
I20260811 12:02:40.354113 13159 leader_election.cc:302] T 00000000000000000000000000000000 P d924d872636d40318a5089eb6fccbf6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d924d872636d40318a5089eb6fccbf6c; no voters:
I20260811 12:02:40.354702 13159 leader_election.cc:288] T 00000000000000000000000000000000 P d924d872636d40318a5089eb6fccbf6c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:02:40.354918 13167 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d924d872636d40318a5089eb6fccbf6c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:02:40.356040 13167 raft_consensus.cc:686] T 00000000000000000000000000000000 P d924d872636d40318a5089eb6fccbf6c [term 1 LEADER]: Becoming Leader. State: Replica: d924d872636d40318a5089eb6fccbf6c, State: Running, Role: LEADER
I20260811 12:02:40.356598 13167 consensus_queue.cc:237] T 00000000000000000000000000000000 P d924d872636d40318a5089eb6fccbf6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d924d872636d40318a5089eb6fccbf6c" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 41979 } }
I20260811 12:02:40.356962 13159 sys_catalog.cc:564] T 00000000000000000000000000000000 P d924d872636d40318a5089eb6fccbf6c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:02:40.359674 13170 sys_catalog.cc:455] T 00000000000000000000000000000000 P d924d872636d40318a5089eb6fccbf6c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d924d872636d40318a5089eb6fccbf6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d924d872636d40318a5089eb6fccbf6c" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 41979 } } }
I20260811 12:02:40.360090 13170 sys_catalog.cc:458] T 00000000000000000000000000000000 P d924d872636d40318a5089eb6fccbf6c [sys.catalog]: This master's current role is: LEADER
I20260811 12:02:40.359783 13171 sys_catalog.cc:455] T 00000000000000000000000000000000 P d924d872636d40318a5089eb6fccbf6c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d924d872636d40318a5089eb6fccbf6c. Latest consensus state: current_term: 1 leader_uuid: "d924d872636d40318a5089eb6fccbf6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d924d872636d40318a5089eb6fccbf6c" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 41979 } } }
I20260811 12:02:40.360391 13171 sys_catalog.cc:458] T 00000000000000000000000000000000 P d924d872636d40318a5089eb6fccbf6c [sys.catalog]: This master's current role is: LEADER
I20260811 12:02:40.363063 13182 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:02:40.368430 13182 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:02:40.377630 13182 catalog_manager.cc:1347] Generated new cluster ID: 463b56577939433cbfb3a299efcf18a9
I20260811 12:02:40.377746 13182 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:02:40.402521 13182 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:02:40.403962 13182 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:02:40.410043 13182 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d924d872636d40318a5089eb6fccbf6c: Generated new TSK 0
I20260811 12:02:40.410832 13182 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.221.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab.
Entry for principal kudu/127.9.221.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab.
I20260811 12:02:40.472663 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.129:0
--local_ip_for_outbound_sockets=127.9.221.129
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:41979
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab
--principal=kudu/127.9.221.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:45699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:02:40.815793 13203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:02:40.816074 13203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:02:40.816262 13203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:02:40.823283 13203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:02:40.823385 13203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:02:40.823554 13203 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 12:02:40.823637 13203 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 12:02:40.823797 13203 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.129
I20260811 12:02:40.833338 13203 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:45699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab
--principal=kudu/127.9.221.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.221.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.221.190:41979
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.129
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:02:40.834609 13203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:02:40.835938 13203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:02:40.849086 13218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:40.849429 13217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:40.849109 13221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:02:40.849906 13203 server_base.cc:1034] running on GCE node
I20260811 12:02:40.850481 13203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:02:40.851352 13203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:02:40.852562 13203 hybrid_clock.cc:648] HybridClock initialized: now 1786449760852519 us; error 43 us; skew 500 ppm
I20260811 12:02:40.852860 13203 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:02:40 dist-test-slave-mjjr krb5kdc[12997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449760, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:02:40.855868 13203 init.cc:377] Logged in from keytab as kudu/127.9.221.129@KRBTEST.COM (short username kudu)
I20260811 12:02:40.857364 13203 webserver.cc:460] Webserver started at http://127.9.221.129:33803/ using document root <none> and password file <none>
I20260811 12:02:40.858104 13203 fs_manager.cc:359] Metadata directory not provided
I20260811 12:02:40.858251 13203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:02:40.858574 13203 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:02:40.861482 13203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-0/data/instance:
uuid: "e286e4695ccd4ff48b17493f78c4629d"
format_stamp: "Formatted at 2026-08-11 12:02:40 on dist-test-slave-mjjr"
server_key: "3c20d9500da3087c2bbe7031e4057a8d"
server_key_iv: "49747dd0f52d63d0a85e7c994efdb672"
server_key_version: "encryptionkey@0"
I20260811 12:02:40.862283 13203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance:
uuid: "e286e4695ccd4ff48b17493f78c4629d"
format_stamp: "Formatted at 2026-08-11 12:02:40 on dist-test-slave-mjjr"
server_key: "3c20d9500da3087c2bbe7031e4057a8d"
server_key_iv: "49747dd0f52d63d0a85e7c994efdb672"
server_key_version: "encryptionkey@0"
I20260811 12:02:40.867280 13203 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 12:02:40.870493 13235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:40.871797 13203 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:02:40.871964 13203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-0/wal
uuid: "e286e4695ccd4ff48b17493f78c4629d"
format_stamp: "Formatted at 2026-08-11 12:02:40 on dist-test-slave-mjjr"
server_key: "3c20d9500da3087c2bbe7031e4057a8d"
server_key_iv: "49747dd0f52d63d0a85e7c994efdb672"
server_key_version: "encryptionkey@0"
I20260811 12:02:40.872156 13203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:02:40.908209 13203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:02:40.912604 13203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:02:40.912892 13203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:02:40.915408 13203 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:02:40.915557 13203 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:40.915704 13203 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:02:40.915810 13203 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 12:02:40 dist-test-slave-mjjr krb5kdc[12997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449760, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:02:40.941586 13251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:40.916190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:55683 (local address 127.9.221.190:41979)
0811 12:02:40.916755 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:40.916763 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:40.917926 (+ 1163us) server_negotiation.cc:407] Connection header received
0811 12:02:40.920414 (+ 2488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:40.920420 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:40.920511 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:40.920642 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:40.922123 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:40.923273 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:40.926668 (+ 3395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:40.927300 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:40.927470 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:40.932628 (+ 5158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:40.932635 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:40.932640 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:40.932703 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:40.937009 (+ 4306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:40.938151 (+ 1142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:40.938157 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:40.938161 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:40.938330 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:40.939059 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:40.939067 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:40.939073 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:40.939498 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:40.939691 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:40.940829 (+ 1138us) server_negotiation.cc:299] Negotiation successful
0811 12:02:40.941118 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":345,"threads_started":1}
I20260811 12:02:40.943318 13252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:40.916857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:41979 (local address 127.9.221.129:55683)
0811 12:02:40.917748 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:40.917802 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:40.920113 (+ 2311us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:40.920887 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:40.920894 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:40.921388 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:02:40.921936 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:40.921952 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:40.923522 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:40.923534 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:40.926494 (+ 2960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:40.926518 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:40.927594 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:40.927604 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:40.927938 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:40.928698 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:02:40.928743 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:02:40.932359 (+ 3616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:02:40.937173 (+ 4814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:40.937187 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:40.937208 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:40.937980 (+ 772us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:40.938469 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:40.938476 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:40.938482 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:40.938863 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:40.939686 (+ 823us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:02:40.939700 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:02:40.940274 (+ 574us) client_negotiation.cc:769] Sending connection context
0811 12:02:40.940668 (+ 394us) client_negotiation.cc:240] Negotiation successful
0811 12:02:40.941387 (+ 719us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":391,"threads_started":1}
I20260811 12:02:40.958010 13203 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.129:37553
I20260811 12:02:40.958045 13426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.129:37553 every 8 connection(s)
I20260811 12:02:40.959457 13203 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
I20260811 12:02:40.962005 13427 heartbeater.cc:347] Connected to a master server at 127.9.221.190:41979
I20260811 12:02:40.962271 13427 heartbeater.cc:464] Registering TS with master...
I20260811 12:02:40.962939 13427 heartbeater.cc:511] Master 127.9.221.190:41979 requested a full tablet report, sending...
I20260811 12:02:40.964244 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 13203
I20260811 12:02:40.964452 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance
I20260811 12:02:40.965283 10102 external_mini_cluster.cc:1428] Setting key 160af37a2789225601945a1bce2f50a7
I20260811 12:02:40.965415 13071 ts_manager.cc:194] Registered new tserver with Master: e286e4695ccd4ff48b17493f78c4629d (127.9.221.129:37553)
I20260811 12:02:40.966420 13251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:40.951287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:56193 (local address 127.9.221.190:41979)
0811 12:02:40.951499 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:40.951507 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:40.952471 (+ 964us) server_negotiation.cc:407] Connection header received
0811 12:02:40.952698 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:40.952706 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:40.952803 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:40.952905 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:40.954365 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:40.955892 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:40.957561 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:40.958299 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:40.958430 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:40.960473 (+ 2043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:40.960481 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:40.960485 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:40.960521 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:40.963544 (+ 3023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:40.964376 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:40.964383 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:40.964386 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:40.964568 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:40.964953 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:40.964958 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:40.964961 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:40.965252 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:40.965419 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:40.965801 (+ 382us) server_negotiation.cc:299] Negotiation successful
0811 12:02:40.966019 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 12:02:40.966799 13406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:40.951418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:41979 (local address 127.9.221.129:56193)
0811 12:02:40.952334 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:40.952360 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:40.952543 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:40.953059 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:40.953062 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:40.953742 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:02:40.954173 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:40.954191 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:40.956025 (+ 1834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:40.956029 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:40.957322 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:40.957355 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:40.958470 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:40.958474 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:40.958597 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:40.959230 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:02:40.959247 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:02:40.960231 (+ 984us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:02:40.963749 (+ 3518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:40.963754 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:40.963756 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:40.964256 (+ 500us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:40.964684 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:40.964690 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:40.964694 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:40.964829 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:40.965370 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:02:40.965375 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:02:40.965549 (+ 174us) client_negotiation.cc:769] Sending connection context
0811 12:02:40.965686 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 12:02:40.966002 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":711,"thread_start_us":510,"threads_started":1}
I20260811 12:02:40.967660 13071 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.129@KRBTEST.COM'} at 127.9.221.129:55683
WARNING: no policy specified for kudu/127.9.221.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab.
Entry for principal kudu/127.9.221.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab.
I20260811 12:02:41.010916 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.130:0
--local_ip_for_outbound_sockets=127.9.221.130
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:41979
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab
--principal=kudu/127.9.221.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:45699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:02:41.337407 13432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:02:41.337706 13432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:02:41.337811 13432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:02:41.345090 13432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:02:41.345197 13432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:02:41.345283 13432 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 12:02:41.345336 13432 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 12:02:41.345436 13432 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.130
I20260811 12:02:41.355167 13432 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:45699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab
--principal=kudu/127.9.221.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.221.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.221.190:41979
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.130
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:02:41.356482 13432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:02:41.357908 13432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:02:41.371084 13442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:41.371117 13440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:41.371186 13439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:02:41.372150 13432 server_base.cc:1034] running on GCE node
I20260811 12:02:41.372752 13432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:02:41.373622 13432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:02:41.374846 13432 hybrid_clock.cc:648] HybridClock initialized: now 1786449761374802 us; error 56 us; skew 500 ppm
I20260811 12:02:41.375151 13432 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:02:41 dist-test-slave-mjjr krb5kdc[12997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449761, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:02:41.378901 13432 init.cc:377] Logged in from keytab as kudu/127.9.221.130@KRBTEST.COM (short username kudu)
I20260811 12:02:41.380424 13432 webserver.cc:460] Webserver started at http://127.9.221.130:33245/ using document root <none> and password file <none>
I20260811 12:02:41.381170 13432 fs_manager.cc:359] Metadata directory not provided
I20260811 12:02:41.381418 13432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:02:41.381773 13432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:02:41.384718 13432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-1/data/instance:
uuid: "4395ee3783a646e09e90f735d8bdee70"
format_stamp: "Formatted at 2026-08-11 12:02:41 on dist-test-slave-mjjr"
server_key: "a83328545bc8a6512a0107eed754a073"
server_key_iv: "999677fed07d6ac650d60b97fc3ff7a8"
server_key_version: "encryptionkey@0"
I20260811 12:02:41.385465 13432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance:
uuid: "4395ee3783a646e09e90f735d8bdee70"
format_stamp: "Formatted at 2026-08-11 12:02:41 on dist-test-slave-mjjr"
server_key: "a83328545bc8a6512a0107eed754a073"
server_key_iv: "999677fed07d6ac650d60b97fc3ff7a8"
server_key_version: "encryptionkey@0"
I20260811 12:02:41.390455 13432 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:02:41.393803 13456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:41.395313 13432 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 12:02:41.395478 13432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-1/wal
uuid: "4395ee3783a646e09e90f735d8bdee70"
format_stamp: "Formatted at 2026-08-11 12:02:41 on dist-test-slave-mjjr"
server_key: "a83328545bc8a6512a0107eed754a073"
server_key_iv: "999677fed07d6ac650d60b97fc3ff7a8"
server_key_version: "encryptionkey@0"
I20260811 12:02:41.395809 13432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:02:41.413879 13432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:02:41.417858 13432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:02:41.418133 13432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:02:41.420853 13432 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:02:41.421001 13432 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:41.421162 13432 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:02:41.421279 13432 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 12:02:41 dist-test-slave-mjjr krb5kdc[12997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449761, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.130@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:02:41.444479 13251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:41.421185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:55449 (local address 127.9.221.190:41979)
0811 12:02:41.421404 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:41.421410 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:41.422959 (+ 1549us) server_negotiation.cc:407] Connection header received
0811 12:02:41.425025 (+ 2066us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:41.425033 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:41.425143 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:41.425293 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:41.427540 (+ 2247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:41.428913 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:41.431613 (+ 2700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:41.432463 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:41.432626 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:41.436573 (+ 3947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:41.436583 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:41.436587 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:41.436619 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:41.440394 (+ 3775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:41.441437 (+ 1043us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:41.441445 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:41.441450 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:41.441618 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:41.442248 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:41.442261 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:41.442265 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:41.442691 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:41.442869 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:41.443845 (+ 976us) server_negotiation.cc:299] Negotiation successful
0811 12:02:41.444100 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 12:02:41.445612 13471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:41.421846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:41979 (local address 127.9.221.130:55449)
0811 12:02:41.422766 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:41.422813 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:41.424745 (+ 1932us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:41.425574 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:41.425584 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:41.426436 (+ 852us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:02:41.427307 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:41.427337 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:41.429113 (+ 1776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:41.429120 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:41.431467 (+ 2347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:41.431483 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:41.432748 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:41.432755 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:41.432937 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:41.433713 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:02:41.433748 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:02:41.436325 (+ 2577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:02:41.440556 (+ 4231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:41.440563 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:41.440583 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:41.441299 (+ 716us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:41.441762 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:41.441766 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:41.441772 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:41.442112 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:41.442878 (+ 766us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:02:41.442888 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:02:41.443315 (+ 427us) client_negotiation.cc:769] Sending connection context
0811 12:02:41.443721 (+ 406us) client_negotiation.cc:240] Negotiation successful
0811 12:02:41.444291 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"thread_start_us":403,"threads_started":1}
I20260811 12:02:41.466818 13251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:41.452026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:33175 (local address 127.9.221.190:41979)
0811 12:02:41.452232 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:41.452239 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:41.453126 (+ 887us) server_negotiation.cc:407] Connection header received
0811 12:02:41.453269 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:41.453277 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:41.453377 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:41.453507 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:41.454366 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:41.455862 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:41.457418 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:41.457922 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:41.458059 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:41.459967 (+ 1908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:41.459972 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:41.459975 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:41.460005 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:41.463603 (+ 3598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:41.464760 (+ 1157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:41.464768 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:41.464771 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:41.464916 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:41.465386 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:41.465390 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:41.465392 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:41.465730 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:41.465852 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:41.466402 (+ 550us) server_negotiation.cc:299] Negotiation successful
0811 12:02:41.466576 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 12:02:41.467455 13578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:41.452190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:41979 (local address 127.9.221.130:33175)
0811 12:02:41.453008 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:41.453035 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:41.453141 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:41.453526 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:41.453529 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:41.453958 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:02:41.454230 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:41.454239 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:41.456004 (+ 1765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:41.456008 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:41.457262 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:41.457274 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:41.458134 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:41.458140 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:41.458296 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:41.458941 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:02:41.458959 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:02:41.459750 (+ 791us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:02:41.463796 (+ 4046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:41.463803 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:41.463809 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:41.464595 (+ 786us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:41.465050 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:41.465058 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:41.465064 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:41.465269 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:41.465855 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:02:41.465863 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:02:41.466122 (+ 259us) client_negotiation.cc:769] Sending connection context
0811 12:02:41.466332 (+ 210us) client_negotiation.cc:240] Negotiation successful
0811 12:02:41.466791 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":672,"thread_start_us":511,"threads_started":1}
I20260811 12:02:41.484874 13432 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.130:38365
I20260811 12:02:41.484915 13658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.130:38365 every 8 connection(s)
I20260811 12:02:41.486129 13432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
I20260811 12:02:41.488269 13666 heartbeater.cc:347] Connected to a master server at 127.9.221.190:41979
I20260811 12:02:41.488498 13666 heartbeater.cc:464] Registering TS with master...
I20260811 12:02:41.489149 13666 heartbeater.cc:511] Master 127.9.221.190:41979 requested a full tablet report, sending...
I20260811 12:02:41.489748 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 13432
I20260811 12:02:41.489959 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance
I20260811 12:02:41.490592 10102 external_mini_cluster.cc:1428] Setting key 8219027e71e28c7b002b2dc4fd7e8a59
I20260811 12:02:41.490595 13071 ts_manager.cc:194] Registered new tserver with Master: 4395ee3783a646e09e90f735d8bdee70 (127.9.221.130:38365)
I20260811 12:02:41.491526 13071 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.130@KRBTEST.COM'} at 127.9.221.130:55449
WARNING: no policy specified for kudu/127.9.221.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab.
Entry for principal kudu/127.9.221.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab.
I20260811 12:02:41.532487 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.131:0
--local_ip_for_outbound_sockets=127.9.221.131
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:41979
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab
--principal=kudu/127.9.221.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:45699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:02:41.869458 13672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:02:41.869899 13672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:02:41.870070 13672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:02:41.879069 13672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:02:41.879184 13672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:02:41.879271 13672 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 12:02:41.879323 13672 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 12:02:41.879392 13672 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.131
I20260811 12:02:41.889462 13672 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:45699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab
--principal=kudu/127.9.221.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.221.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.221.190:41979
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.131
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:02:41.890740 13672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:02:41.892113 13672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:02:41.905438 13683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:41.905339 13686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:41.905398 13684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:02:41.906435 13672 server_base.cc:1034] running on GCE node
I20260811 12:02:41.907055 13672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:02:41.908059 13672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:02:41.909256 13672 hybrid_clock.cc:648] HybridClock initialized: now 1786449761909198 us; error 52 us; skew 500 ppm
I20260811 12:02:41.909572 13672 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:02:41 dist-test-slave-mjjr krb5kdc[12997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449761, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:02:41.912694 13672 init.cc:377] Logged in from keytab as kudu/127.9.221.131@KRBTEST.COM (short username kudu)
I20260811 12:02:41.914148 13672 webserver.cc:460] Webserver started at http://127.9.221.131:46601/ using document root <none> and password file <none>
I20260811 12:02:41.914820 13672 fs_manager.cc:359] Metadata directory not provided
I20260811 12:02:41.914948 13672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:02:41.915293 13672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:02:41.918227 13672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-2/data/instance:
uuid: "8967cd6438d942b89dedf51a53d1f081"
format_stamp: "Formatted at 2026-08-11 12:02:41 on dist-test-slave-mjjr"
server_key: "24948f9c785c9640b4b0752b4ade285a"
server_key_iv: "f009cbbf5985403c4de4747421fe183a"
server_key_version: "encryptionkey@0"
I20260811 12:02:41.918991 13672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance:
uuid: "8967cd6438d942b89dedf51a53d1f081"
format_stamp: "Formatted at 2026-08-11 12:02:41 on dist-test-slave-mjjr"
server_key: "24948f9c785c9640b4b0752b4ade285a"
server_key_iv: "f009cbbf5985403c4de4747421fe183a"
server_key_version: "encryptionkey@0"
I20260811 12:02:41.924158 13672 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260811 12:02:41.927440 13694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:41.928763 13672 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:02:41.928944 13672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-2/wal
uuid: "8967cd6438d942b89dedf51a53d1f081"
format_stamp: "Formatted at 2026-08-11 12:02:41 on dist-test-slave-mjjr"
server_key: "24948f9c785c9640b4b0752b4ade285a"
server_key_iv: "f009cbbf5985403c4de4747421fe183a"
server_key_version: "encryptionkey@0"
I20260811 12:02:41.929167 13672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:02:41.955901 13672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:02:41.960471 13672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:02:41.960770 13672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:02:41.963737 13672 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:02:41.963861 13672 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:41.963999 13672 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:02:41.964121 13672 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:41.970834 13427 heartbeater.cc:503] Master 127.9.221.190:41979 was elected leader, sending a full tablet report...
Aug 11 12:02:41 dist-test-slave-mjjr krb5kdc[12997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449761, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.131@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:02:41.986889 13251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:41.964279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:48145 (local address 127.9.221.190:41979)
0811 12:02:41.964492 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:41.964500 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:41.966196 (+ 1696us) server_negotiation.cc:407] Connection header received
0811 12:02:41.968321 (+ 2125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:41.968328 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:41.968419 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:41.968530 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:41.970562 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:41.971475 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:41.973992 (+ 2517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:41.974549 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:41.974684 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:41.979484 (+ 4800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:41.979491 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:41.979501 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:41.979533 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:41.983026 (+ 3493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:41.984099 (+ 1073us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:41.984107 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:41.984113 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:41.984269 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:41.984726 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:41.984731 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:41.984735 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:41.985093 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:41.985253 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:41.986279 (+ 1026us) server_negotiation.cc:299] Negotiation successful
0811 12:02:41.986524 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 12:02:41.988027 13707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:41.965005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:41979 (local address 127.9.221.131:48145)
0811 12:02:41.966035 (+ 1030us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:41.966077 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:41.968037 (+ 1960us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:41.968761 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:41.968769 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:41.969505 (+ 736us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:02:41.970389 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:41.970407 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:41.971685 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:41.971690 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:41.973818 (+ 2128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:41.973829 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:41.975006 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:41.975016 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:41.975406 (+ 390us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:41.976370 (+ 964us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:02:41.976442 (+ 72us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:02:41.979267 (+ 2825us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:02:41.983165 (+ 3898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:41.983174 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:41.983192 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:41.983921 (+ 729us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:41.984395 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:41.984400 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:41.984404 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:41.984601 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:41.985320 (+ 719us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:02:41.985338 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:02:41.985805 (+ 467us) client_negotiation.cc:769] Sending connection context
0811 12:02:41.986138 (+ 333us) client_negotiation.cc:240] Negotiation successful
0811 12:02:41.986698 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":770,"thread_start_us":475,"threads_started":1}
I20260811 12:02:42.008595 13251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:41.994027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:44749 (local address 127.9.221.190:41979)
0811 12:02:41.994173 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:41.994176 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:41.994902 (+ 726us) server_negotiation.cc:407] Connection header received
0811 12:02:41.995087 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:41.995099 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:41.995164 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:41.995265 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:41.996523 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:41.997538 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:41.999574 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.000257 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.000408 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:42.002245 (+ 1837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:42.002251 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:42.002256 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:42.002294 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:42.005344 (+ 3050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:42.006259 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:42.006263 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:42.006266 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:42.006415 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:42.006931 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:42.006934 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:42.006936 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:42.007322 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:42.007513 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:42.007980 (+ 467us) server_negotiation.cc:299] Negotiation successful
0811 12:02:42.008201 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 12:02:42.008757 13809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:41.994156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:41979 (local address 127.9.221.131:44749)
0811 12:02:41.994800 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:41.994819 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:41.994974 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:41.995289 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:41.995293 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:41.995813 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:02:41.996324 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:41.996345 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:41.997739 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:41.997743 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:41.999338 (+ 1595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:41.999357 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.000452 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.000459 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:42.000683 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:42.001329 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:02:42.001346 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:02:42.002063 (+ 717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:02:42.005517 (+ 3454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:42.005524 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:42.005528 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:42.006128 (+ 600us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:42.006593 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:42.006598 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:42.006604 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:42.006784 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:42.007477 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:02:42.007485 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:02:42.007738 (+ 253us) client_negotiation.cc:769] Sending connection context
0811 12:02:42.007888 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 12:02:42.008238 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":384,"threads_started":1}
I20260811 12:02:42.016235 13672 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.131:41221
I20260811 12:02:42.016275 13914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.131:41221 every 8 connection(s)
I20260811 12:02:42.017409 13672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
I20260811 12:02:42.019479 13916 heartbeater.cc:347] Connected to a master server at 127.9.221.190:41979
I20260811 12:02:42.019734 13916 heartbeater.cc:464] Registering TS with master...
I20260811 12:02:42.020323 13916 heartbeater.cc:511] Master 127.9.221.190:41979 requested a full tablet report, sending...
I20260811 12:02:42.021317 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 13672
I20260811 12:02:42.021519 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance
I20260811 12:02:42.021700 13070 ts_manager.cc:194] Registered new tserver with Master: 8967cd6438d942b89dedf51a53d1f081 (127.9.221.131:41221)
I20260811 12:02:42.022101 10102 external_mini_cluster.cc:1428] Setting key 0ebea5b65276bc6a9e9a5f0160f40270
I20260811 12:02:42.022811 13070 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.131@KRBTEST.COM'} at 127.9.221.131:48145
I20260811 12:02:42.025146 10102 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 12:02:42 dist-test-slave-mjjr krb5kdc[12997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449762, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 12:02:42 dist-test-slave-mjjr krb5kdc[12997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449762, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:02:42.074021 13251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:42.060423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39420 (local address 127.9.221.190:41979)
0811 12:02:42.060696 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:42.060702 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:42.060933 (+ 231us) server_negotiation.cc:407] Connection header received
0811 12:02:42.061095 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:42.061099 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:42.061164 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:42.061251 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:42.062633 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.063437 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.064660 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.065162 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.065258 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:42.068968 (+ 3710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:42.068974 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:42.068977 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:42.069009 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:42.071704 (+ 2695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:42.072327 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:42.072331 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:42.072333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:42.072463 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:42.072773 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:42.072776 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:42.072778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:42.073029 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:42.073204 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:42.073448 (+ 244us) server_negotiation.cc:299] Negotiation successful
0811 12:02:42.073623 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260811 12:02:42.078282 13071 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39420:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:02:42.085090 13071 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:02:42.107478 13937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:42.097192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58036 (local address 127.9.221.130:38365)
0811 12:02:42.097955 (+ 763us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:42.097965 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:42.098840 (+ 875us) server_negotiation.cc:407] Connection header received
0811 12:02:42.099227 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:42.099239 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:42.099491 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:42.100404 (+ 913us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:02:42.100566 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.101970 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.104098 (+ 2128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.105463 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.105596 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:42.105829 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:42.107038 (+ 1209us) server_negotiation.cc:299] Negotiation successful
0811 12:02:42.107196 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":264,"threads_started":1}
I20260811 12:02:42.107517 13934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:42.097035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.130:38365 (local address 127.0.0.1:58036)
0811 12:02:42.098683 (+ 1648us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:42.098715 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:42.098922 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:42.099796 (+ 874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:42.099806 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:42.099834 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:02:42.100216 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:42.100226 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.102179 (+ 1953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.102187 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:42.103868 (+ 1681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:42.103888 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.105986 (+ 2098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.105994 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:42.106812 (+ 818us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:42.106849 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 12:02:42.106996 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 12:02:42.107177 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1258,"thread_start_us":1122,"threads_started":1}
I20260811 12:02:42.111716 13538 tablet_service.cc:1467] Processing CreateTablet for tablet 43e05a60bf144ca4aae527ab1a290f72 (DEFAULT_TABLE table=test-table [id=b0bfa997d98943269a7403e22346ba33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:02:42.112953 13945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:42.097764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:37553 (local address 127.0.0.1:43408)
0811 12:02:42.099491 (+ 1727us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:42.099510 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:42.099719 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:42.106068 (+ 6349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:42.106072 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:42.106106 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:02:42.106630 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:42.106642 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.108709 (+ 2067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.108714 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:42.110780 (+ 2066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:42.110795 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.112134 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.112140 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:42.112316 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:42.112356 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 12:02:42.112480 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 12:02:42.112619 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1605,"thread_start_us":8120,"threads_started":1}
I20260811 12:02:42.113297 13538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43e05a60bf144ca4aae527ab1a290f72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:02:42.113281 13941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:42.098358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43408 (local address 127.9.221.129:37553)
0811 12:02:42.105212 (+ 6854us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:42.105220 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:42.105248 (+ 28us) server_negotiation.cc:407] Connection header received
0811 12:02:42.105345 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:42.105352 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:42.105609 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:42.106208 (+ 599us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:02:42.106792 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.108550 (+ 1758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.110939 (+ 2389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.111917 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.112085 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:42.112290 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:42.112583 (+ 293us) server_negotiation.cc:299] Negotiation successful
0811 12:02:42.112744 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5676,"thread_start_us":5557,"threads_started":1}
I20260811 12:02:42.117280 13310 tablet_service.cc:1467] Processing CreateTablet for tablet 43e05a60bf144ca4aae527ab1a290f72 (DEFAULT_TABLE table=test-table [id=b0bfa997d98943269a7403e22346ba33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:02:42.117962 13940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:42.099004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.131:41221 (local address 127.0.0.1:45438)
0811 12:02:42.106972 (+ 7968us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:42.106991 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:42.107111 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:42.108060 (+ 949us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:42.108065 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:42.108088 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:02:42.108387 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:42.108403 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.113216 (+ 4813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.113222 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:42.115330 (+ 2108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:42.115347 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.117159 (+ 1812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.117166 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:42.117334 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:42.117367 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 12:02:42.117488 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 12:02:42.117629 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7792,"thread_start_us":725,"threads_started":1}
I20260811 12:02:42.118261 13946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:42.099466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45438 (local address 127.9.221.131:41221)
0811 12:02:42.106650 (+ 7184us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:42.106656 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:42.107231 (+ 575us) server_negotiation.cc:407] Connection header received
0811 12:02:42.107327 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:42.107335 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:42.107545 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:42.107749 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:02:42.108637 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.113039 (+ 4402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.115521 (+ 2482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.116941 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.117086 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:42.117308 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:42.117535 (+ 227us) server_negotiation.cc:299] Negotiation successful
0811 12:02:42.117693 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7050,"thread_start_us":6869,"threads_started":1}
I20260811 12:02:42.118736 13310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43e05a60bf144ca4aae527ab1a290f72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:02:42.121862 13781 tablet_service.cc:1467] Processing CreateTablet for tablet 43e05a60bf144ca4aae527ab1a290f72 (DEFAULT_TABLE table=test-table [id=b0bfa997d98943269a7403e22346ba33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:02:42.123262 13781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43e05a60bf144ca4aae527ab1a290f72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:02:42.127919 13947 tablet_bootstrap.cc:492] T 43e05a60bf144ca4aae527ab1a290f72 P 4395ee3783a646e09e90f735d8bdee70: Bootstrap starting.
I20260811 12:02:42.131055 13949 tablet_bootstrap.cc:492] T 43e05a60bf144ca4aae527ab1a290f72 P e286e4695ccd4ff48b17493f78c4629d: Bootstrap starting.
I20260811 12:02:42.131587 13947 tablet_bootstrap.cc:654] T 43e05a60bf144ca4aae527ab1a290f72 P 4395ee3783a646e09e90f735d8bdee70: Neither blocks nor log segments found. Creating new log.
I20260811 12:02:42.132786 13947 log.cc:824] T 43e05a60bf144ca4aae527ab1a290f72 P 4395ee3783a646e09e90f735d8bdee70: Log is configured to *not* fsync() on all Append() calls
I20260811 12:02:42.134569 13949 tablet_bootstrap.cc:654] T 43e05a60bf144ca4aae527ab1a290f72 P e286e4695ccd4ff48b17493f78c4629d: Neither blocks nor log segments found. Creating new log.
I20260811 12:02:42.135811 13949 log.cc:824] T 43e05a60bf144ca4aae527ab1a290f72 P e286e4695ccd4ff48b17493f78c4629d: Log is configured to *not* fsync() on all Append() calls
I20260811 12:02:42.136194 13947 tablet_bootstrap.cc:492] T 43e05a60bf144ca4aae527ab1a290f72 P 4395ee3783a646e09e90f735d8bdee70: No bootstrap required, opened a new log
I20260811 12:02:42.136459 13947 ts_tablet_manager.cc:1397] T 43e05a60bf144ca4aae527ab1a290f72 P 4395ee3783a646e09e90f735d8bdee70: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260811 12:02:42.137292 13957 tablet_bootstrap.cc:492] T 43e05a60bf144ca4aae527ab1a290f72 P 8967cd6438d942b89dedf51a53d1f081: Bootstrap starting.
I20260811 12:02:42.140520 13949 tablet_bootstrap.cc:492] T 43e05a60bf144ca4aae527ab1a290f72 P e286e4695ccd4ff48b17493f78c4629d: No bootstrap required, opened a new log
I20260811 12:02:42.140798 13949 ts_tablet_manager.cc:1397] T 43e05a60bf144ca4aae527ab1a290f72 P e286e4695ccd4ff48b17493f78c4629d: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260811 12:02:42.140843 13957 tablet_bootstrap.cc:654] T 43e05a60bf144ca4aae527ab1a290f72 P 8967cd6438d942b89dedf51a53d1f081: Neither blocks nor log segments found. Creating new log.
I20260811 12:02:42.142038 13957 log.cc:824] T 43e05a60bf144ca4aae527ab1a290f72 P 8967cd6438d942b89dedf51a53d1f081: Log is configured to *not* fsync() on all Append() calls
I20260811 12:02:42.145017 13957 tablet_bootstrap.cc:492] T 43e05a60bf144ca4aae527ab1a290f72 P 8967cd6438d942b89dedf51a53d1f081: No bootstrap required, opened a new log
I20260811 12:02:42.145264 13957 ts_tablet_manager.cc:1397] T 43e05a60bf144ca4aae527ab1a290f72 P 8967cd6438d942b89dedf51a53d1f081: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.005s
I20260811 12:02:42.146994 13947 raft_consensus.cc:348] T 43e05a60bf144ca4aae527ab1a290f72 P 4395ee3783a646e09e90f735d8bdee70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } } peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } }
I20260811 12:02:42.147435 13947 raft_consensus.cc:374] T 43e05a60bf144ca4aae527ab1a290f72 P 4395ee3783a646e09e90f735d8bdee70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:02:42.147539 13947 raft_consensus.cc:729] T 43e05a60bf144ca4aae527ab1a290f72 P 4395ee3783a646e09e90f735d8bdee70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4395ee3783a646e09e90f735d8bdee70, State: Initialized, Role: FOLLOWER
I20260811 12:02:42.148157 13947 consensus_queue.cc:260] T 43e05a60bf144ca4aae527ab1a290f72 P 4395ee3783a646e09e90f735d8bdee70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } } peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } }
I20260811 12:02:42.149272 13666 heartbeater.cc:503] Master 127.9.221.190:41979 was elected leader, sending a full tablet report...
I20260811 12:02:42.149534 13949 raft_consensus.cc:348] T 43e05a60bf144ca4aae527ab1a290f72 P e286e4695ccd4ff48b17493f78c4629d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } } peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } }
I20260811 12:02:42.149871 13949 raft_consensus.cc:374] T 43e05a60bf144ca4aae527ab1a290f72 P e286e4695ccd4ff48b17493f78c4629d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:02:42.150000 13949 raft_consensus.cc:729] T 43e05a60bf144ca4aae527ab1a290f72 P e286e4695ccd4ff48b17493f78c4629d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e286e4695ccd4ff48b17493f78c4629d, State: Initialized, Role: FOLLOWER
I20260811 12:02:42.150591 13949 consensus_queue.cc:260] T 43e05a60bf144ca4aae527ab1a290f72 P e286e4695ccd4ff48b17493f78c4629d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } } peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } }
I20260811 12:02:42.151219 13947 ts_tablet_manager.cc:1428] T 43e05a60bf144ca4aae527ab1a290f72 P 4395ee3783a646e09e90f735d8bdee70: Time spent starting tablet: real 0.015s user 0.012s sys 0.001s
I20260811 12:02:42.152797 13949 ts_tablet_manager.cc:1428] T 43e05a60bf144ca4aae527ab1a290f72 P e286e4695ccd4ff48b17493f78c4629d: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260811 12:02:42.155603 13957 raft_consensus.cc:348] T 43e05a60bf144ca4aae527ab1a290f72 P 8967cd6438d942b89dedf51a53d1f081 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } } peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } }
I20260811 12:02:42.156035 13957 raft_consensus.cc:374] T 43e05a60bf144ca4aae527ab1a290f72 P 8967cd6438d942b89dedf51a53d1f081 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:02:42.156140 13957 raft_consensus.cc:729] T 43e05a60bf144ca4aae527ab1a290f72 P 8967cd6438d942b89dedf51a53d1f081 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8967cd6438d942b89dedf51a53d1f081, State: Initialized, Role: FOLLOWER
I20260811 12:02:42.156729 13957 consensus_queue.cc:260] T 43e05a60bf144ca4aae527ab1a290f72 P 8967cd6438d942b89dedf51a53d1f081 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } } peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } }
I20260811 12:02:42.157809 13916 heartbeater.cc:503] Master 127.9.221.190:41979 was elected leader, sending a full tablet report...
I20260811 12:02:42.158375 13957 ts_tablet_manager.cc:1428] T 43e05a60bf144ca4aae527ab1a290f72 P 8967cd6438d942b89dedf51a53d1f081: Time spent starting tablet: real 0.013s user 0.011s sys 0.003s
W20260811 12:02:42.212152 13428 tablet.cc:2369] T 43e05a60bf144ca4aae527ab1a290f72 P e286e4695ccd4ff48b17493f78c4629d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:02:42.238209 13667 tablet.cc:2369] T 43e05a60bf144ca4aae527ab1a290f72 P 4395ee3783a646e09e90f735d8bdee70: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:02:42.269377 13917 tablet.cc:2369] T 43e05a60bf144ca4aae527ab1a290f72 P 8967cd6438d942b89dedf51a53d1f081: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:02:42.342828 13959 raft_consensus.cc:482] T 43e05a60bf144ca4aae527ab1a290f72 P 4395ee3783a646e09e90f735d8bdee70 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:02:42.343163 13959 raft_consensus.cc:504] T 43e05a60bf144ca4aae527ab1a290f72 P 4395ee3783a646e09e90f735d8bdee70 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } } peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } }
I20260811 12:02:42.344884 13959 leader_election.cc:288] T 43e05a60bf144ca4aae527ab1a290f72 P 4395ee3783a646e09e90f735d8bdee70 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8967cd6438d942b89dedf51a53d1f081 (127.9.221.131:41221), e286e4695ccd4ff48b17493f78c4629d (127.9.221.129:37553)
I20260811 12:02:42.354311 13941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:42.345451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:45147 (local address 127.9.221.129:37553)
0811 12:02:42.345682 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:42.345689 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:42.346479 (+ 790us) server_negotiation.cc:407] Connection header received
0811 12:02:42.346763 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:42.346768 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:42.346852 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:42.346962 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:02:42.347788 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.348936 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.352013 (+ 3077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.353208 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.353376 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:42.353447 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:42.353919 (+ 472us) server_negotiation.cc:299] Negotiation successful
0811 12:02:42.354070 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 12:02:42.354312 13946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:42.345513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:60315 (local address 127.9.221.131:41221)
0811 12:02:42.345729 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:42.345738 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:42.346556 (+ 818us) server_negotiation.cc:407] Connection header received
0811 12:02:42.346759 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:42.346764 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:42.346859 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:42.346954 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:02:42.347846 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.348929 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.352006 (+ 3077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.353210 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.353371 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:42.353452 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:42.353919 (+ 467us) server_negotiation.cc:299] Negotiation successful
0811 12:02:42.354061 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 12:02:42.354593 13963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:42.345373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.131:41221 (local address 127.9.221.130:60315)
0811 12:02:42.346428 (+ 1055us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:42.346444 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:42.346621 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:42.346984 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:42.346989 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:42.347017 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:02:42.347689 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:42.347703 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.349068 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.349072 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:42.351843 (+ 2771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:42.351859 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.353510 (+ 1651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.353516 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:42.353727 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:42.353781 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 12:02:42.353878 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 12:02:42.354095 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":929,"mutex_wait_us":259,"thread_start_us":455,"threads_started":1}
I20260811 12:02:42.354887 13962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:42.345358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:37553 (local address 127.9.221.130:45147)
0811 12:02:42.346349 (+ 991us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:42.346369 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:42.346558 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:42.346984 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:42.346989 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:42.347079 (+ 90us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:02:42.347576 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:42.347621 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.349067 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.349072 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:42.351843 (+ 2771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:42.351859 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.353516 (+ 1657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.353522 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:42.353731 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:42.353781 (+ 50us) client_negotiation.cc:769] Sending connection context
0811 12:02:42.353878 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 12:02:42.354095 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":838,"mutex_wait_us":253,"spinlock_wait_cycles":20736,"thread_start_us":694,"threads_started":1}
I20260811 12:02:42.355628 13823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43e05a60bf144ca4aae527ab1a290f72" candidate_uuid: "4395ee3783a646e09e90f735d8bdee70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8967cd6438d942b89dedf51a53d1f081" is_pre_election: true
I20260811 12:02:42.355942 13344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43e05a60bf144ca4aae527ab1a290f72" candidate_uuid: "4395ee3783a646e09e90f735d8bdee70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e286e4695ccd4ff48b17493f78c4629d" is_pre_election: true
I20260811 12:02:42.356259 13823 raft_consensus.cc:2454] T 43e05a60bf144ca4aae527ab1a290f72 P 8967cd6438d942b89dedf51a53d1f081 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4395ee3783a646e09e90f735d8bdee70 in term 0.
I20260811 12:02:42.356504 13344 raft_consensus.cc:2454] T 43e05a60bf144ca4aae527ab1a290f72 P e286e4695ccd4ff48b17493f78c4629d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4395ee3783a646e09e90f735d8bdee70 in term 0.
I20260811 12:02:42.357052 13464 leader_election.cc:302] T 43e05a60bf144ca4aae527ab1a290f72 P 4395ee3783a646e09e90f735d8bdee70 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4395ee3783a646e09e90f735d8bdee70, 8967cd6438d942b89dedf51a53d1f081; no voters:
I20260811 12:02:42.357419 13959 raft_consensus.cc:2781] T 43e05a60bf144ca4aae527ab1a290f72 P 4395ee3783a646e09e90f735d8bdee70 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:02:42.357614 13959 raft_consensus.cc:482] T 43e05a60bf144ca4aae527ab1a290f72 P 4395ee3783a646e09e90f735d8bdee70 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:02:42.357729 13959 raft_consensus.cc:3037] T 43e05a60bf144ca4aae527ab1a290f72 P 4395ee3783a646e09e90f735d8bdee70 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:02:42.360219 13959 raft_consensus.cc:504] T 43e05a60bf144ca4aae527ab1a290f72 P 4395ee3783a646e09e90f735d8bdee70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } } peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } }
I20260811 12:02:42.360834 13959 leader_election.cc:288] T 43e05a60bf144ca4aae527ab1a290f72 P 4395ee3783a646e09e90f735d8bdee70 [CANDIDATE]: Term 1 election: Requested vote from peers 8967cd6438d942b89dedf51a53d1f081 (127.9.221.131:41221), e286e4695ccd4ff48b17493f78c4629d (127.9.221.129:37553)
I20260811 12:02:42.361279 13823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43e05a60bf144ca4aae527ab1a290f72" candidate_uuid: "4395ee3783a646e09e90f735d8bdee70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8967cd6438d942b89dedf51a53d1f081"
I20260811 12:02:42.361334 13344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43e05a60bf144ca4aae527ab1a290f72" candidate_uuid: "4395ee3783a646e09e90f735d8bdee70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e286e4695ccd4ff48b17493f78c4629d"
I20260811 12:02:42.361460 13823 raft_consensus.cc:3037] T 43e05a60bf144ca4aae527ab1a290f72 P 8967cd6438d942b89dedf51a53d1f081 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:02:42.361573 13344 raft_consensus.cc:3037] T 43e05a60bf144ca4aae527ab1a290f72 P e286e4695ccd4ff48b17493f78c4629d [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:02:42.363763 13823 raft_consensus.cc:2454] T 43e05a60bf144ca4aae527ab1a290f72 P 8967cd6438d942b89dedf51a53d1f081 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4395ee3783a646e09e90f735d8bdee70 in term 1.
I20260811 12:02:42.364023 13344 raft_consensus.cc:2454] T 43e05a60bf144ca4aae527ab1a290f72 P e286e4695ccd4ff48b17493f78c4629d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4395ee3783a646e09e90f735d8bdee70 in term 1.
I20260811 12:02:42.364181 13464 leader_election.cc:302] T 43e05a60bf144ca4aae527ab1a290f72 P 4395ee3783a646e09e90f735d8bdee70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4395ee3783a646e09e90f735d8bdee70, 8967cd6438d942b89dedf51a53d1f081; no voters:
I20260811 12:02:42.364519 13959 raft_consensus.cc:2781] T 43e05a60bf144ca4aae527ab1a290f72 P 4395ee3783a646e09e90f735d8bdee70 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:02:42.364900 13959 raft_consensus.cc:686] T 43e05a60bf144ca4aae527ab1a290f72 P 4395ee3783a646e09e90f735d8bdee70 [term 1 LEADER]: Becoming Leader. State: Replica: 4395ee3783a646e09e90f735d8bdee70, State: Running, Role: LEADER
I20260811 12:02:42.365433 13959 consensus_queue.cc:237] T 43e05a60bf144ca4aae527ab1a290f72 P 4395ee3783a646e09e90f735d8bdee70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } } peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } }
I20260811 12:02:42.369961 13070 catalog_manager.cc:5495] T 43e05a60bf144ca4aae527ab1a290f72 P 4395ee3783a646e09e90f735d8bdee70 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4395ee3783a646e09e90f735d8bdee70 (127.9.221.130). New cstate: current_term: 1 leader_uuid: "4395ee3783a646e09e90f735d8bdee70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } health_report { overall_health: UNKNOWN } } }
Aug 11 12:02:42 dist-test-slave-mjjr krb5kdc[12997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449760, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:02:42.435789 13976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:42.422422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:41979 (local address 127.0.0.1:39430)
0811 12:02:42.423045 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:42.423061 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:42.423186 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:42.423519 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:42.423522 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:42.424305 (+ 783us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:02:42.424566 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:42.424575 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.425601 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.425608 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:42.426565 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:42.426576 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.427389 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.427393 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:42.427515 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:42.428254 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:02:42.428271 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:02:42.430371 (+ 2100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:02:42.433317 (+ 2946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:42.433327 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:42.433332 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:42.433836 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:42.434193 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:42.434196 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:42.434198 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:42.434338 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:42.434907 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:02:42.434913 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:02:42.435045 (+ 132us) client_negotiation.cc:769] Sending connection context
0811 12:02:42.435177 (+ 132us) client_negotiation.cc:240] Negotiation successful
0811 12:02:42.435363 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":338,"threads_started":1}
I20260811 12:02:42.435832 13251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:42.422349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39430 (local address 127.9.221.190:41979)
0811 12:02:42.422568 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:42.422575 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:42.423153 (+ 578us) server_negotiation.cc:407] Connection header received
0811 12:02:42.423290 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:42.423293 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:42.423363 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:42.423465 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:42.424725 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.425485 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.426717 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.427232 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.427327 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:42.430535 (+ 3208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:42.430540 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:42.430543 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:42.430572 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:42.433182 (+ 2610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:42.433949 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:42.433960 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:42.433963 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:42.434096 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:42.434456 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:42.434460 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:42.434462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:42.434784 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:42.434909 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:42.435281 (+ 372us) server_negotiation.cc:299] Negotiation successful
0811 12:02:42.435466 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":80}
I20260811 12:02:42.440555 13070 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:39430:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 12:02:42.441612 13070 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:02:42.448506 13310 tablet_service.cc:1467] Processing CreateTablet for tablet 0917901cd1074e90a8d3bb266d540e30 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b261b6b9df3345d7ab7e3482a463f1a6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 12:02:42.449064 13781 tablet_service.cc:1467] Processing CreateTablet for tablet 0917901cd1074e90a8d3bb266d540e30 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b261b6b9df3345d7ab7e3482a463f1a6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 12:02:42.449067 13538 tablet_service.cc:1467] Processing CreateTablet for tablet 0917901cd1074e90a8d3bb266d540e30 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b261b6b9df3345d7ab7e3482a463f1a6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 12:02:42.449666 13310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0917901cd1074e90a8d3bb266d540e30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:02:42.449856 13781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0917901cd1074e90a8d3bb266d540e30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:02:42.449976 13538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0917901cd1074e90a8d3bb266d540e30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:02:42.455511 13957 tablet_bootstrap.cc:492] T 0917901cd1074e90a8d3bb266d540e30 P 8967cd6438d942b89dedf51a53d1f081: Bootstrap starting.
I20260811 12:02:42.457007 13957 tablet_bootstrap.cc:654] T 0917901cd1074e90a8d3bb266d540e30 P 8967cd6438d942b89dedf51a53d1f081: Neither blocks nor log segments found. Creating new log.
I20260811 12:02:42.457508 13949 tablet_bootstrap.cc:492] T 0917901cd1074e90a8d3bb266d540e30 P e286e4695ccd4ff48b17493f78c4629d: Bootstrap starting.
I20260811 12:02:42.457616 13947 tablet_bootstrap.cc:492] T 0917901cd1074e90a8d3bb266d540e30 P 4395ee3783a646e09e90f735d8bdee70: Bootstrap starting.
I20260811 12:02:42.459223 13949 tablet_bootstrap.cc:654] T 0917901cd1074e90a8d3bb266d540e30 P e286e4695ccd4ff48b17493f78c4629d: Neither blocks nor log segments found. Creating new log.
I20260811 12:02:42.459458 13947 tablet_bootstrap.cc:654] T 0917901cd1074e90a8d3bb266d540e30 P 4395ee3783a646e09e90f735d8bdee70: Neither blocks nor log segments found. Creating new log.
I20260811 12:02:42.459491 13957 tablet_bootstrap.cc:492] T 0917901cd1074e90a8d3bb266d540e30 P 8967cd6438d942b89dedf51a53d1f081: No bootstrap required, opened a new log
I20260811 12:02:42.459648 13957 ts_tablet_manager.cc:1397] T 0917901cd1074e90a8d3bb266d540e30 P 8967cd6438d942b89dedf51a53d1f081: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 12:02:42.460536 13957 raft_consensus.cc:348] T 0917901cd1074e90a8d3bb266d540e30 P 8967cd6438d942b89dedf51a53d1f081 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } }
I20260811 12:02:42.460821 13957 raft_consensus.cc:374] T 0917901cd1074e90a8d3bb266d540e30 P 8967cd6438d942b89dedf51a53d1f081 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:02:42.460918 13957 raft_consensus.cc:729] T 0917901cd1074e90a8d3bb266d540e30 P 8967cd6438d942b89dedf51a53d1f081 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8967cd6438d942b89dedf51a53d1f081, State: Initialized, Role: FOLLOWER
I20260811 12:02:42.461167 13957 consensus_queue.cc:260] T 0917901cd1074e90a8d3bb266d540e30 P 8967cd6438d942b89dedf51a53d1f081 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } }
I20260811 12:02:42.461848 13949 tablet_bootstrap.cc:492] T 0917901cd1074e90a8d3bb266d540e30 P e286e4695ccd4ff48b17493f78c4629d: No bootstrap required, opened a new log
I20260811 12:02:42.461992 13949 ts_tablet_manager.cc:1397] T 0917901cd1074e90a8d3bb266d540e30 P e286e4695ccd4ff48b17493f78c4629d: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 12:02:42.461781 13961 tablet_replica.cc:439] T 0917901cd1074e90a8d3bb266d540e30 P 8967cd6438d942b89dedf51a53d1f081: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } } }
I20260811 12:02:42.462219 13961 tablet_replica.cc:442] T 0917901cd1074e90a8d3bb266d540e30 P 8967cd6438d942b89dedf51a53d1f081: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:02:42.461809 13957 ts_tablet_manager.cc:1428] T 0917901cd1074e90a8d3bb266d540e30 P 8967cd6438d942b89dedf51a53d1f081: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 12:02:42.462836 13949 raft_consensus.cc:348] T 0917901cd1074e90a8d3bb266d540e30 P e286e4695ccd4ff48b17493f78c4629d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } }
I20260811 12:02:42.463097 13949 raft_consensus.cc:374] T 0917901cd1074e90a8d3bb266d540e30 P e286e4695ccd4ff48b17493f78c4629d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:02:42.463207 13949 raft_consensus.cc:729] T 0917901cd1074e90a8d3bb266d540e30 P e286e4695ccd4ff48b17493f78c4629d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e286e4695ccd4ff48b17493f78c4629d, State: Initialized, Role: FOLLOWER
I20260811 12:02:42.463529 13947 tablet_bootstrap.cc:492] T 0917901cd1074e90a8d3bb266d540e30 P 4395ee3783a646e09e90f735d8bdee70: No bootstrap required, opened a new log
I20260811 12:02:42.463456 13949 consensus_queue.cc:260] T 0917901cd1074e90a8d3bb266d540e30 P e286e4695ccd4ff48b17493f78c4629d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } }
I20260811 12:02:42.463714 13947 ts_tablet_manager.cc:1397] T 0917901cd1074e90a8d3bb266d540e30 P 4395ee3783a646e09e90f735d8bdee70: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260811 12:02:42.464109 13949 ts_tablet_manager.cc:1428] T 0917901cd1074e90a8d3bb266d540e30 P e286e4695ccd4ff48b17493f78c4629d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 12:02:42.464475 13960 tablet_replica.cc:439] T 0917901cd1074e90a8d3bb266d540e30 P e286e4695ccd4ff48b17493f78c4629d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } } }
I20260811 12:02:42.464658 13947 raft_consensus.cc:348] T 0917901cd1074e90a8d3bb266d540e30 P 4395ee3783a646e09e90f735d8bdee70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } }
I20260811 12:02:42.464895 13960 tablet_replica.cc:442] T 0917901cd1074e90a8d3bb266d540e30 P e286e4695ccd4ff48b17493f78c4629d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:02:42.464883 13947 raft_consensus.cc:374] T 0917901cd1074e90a8d3bb266d540e30 P 4395ee3783a646e09e90f735d8bdee70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:02:42.464993 13947 raft_consensus.cc:729] T 0917901cd1074e90a8d3bb266d540e30 P 4395ee3783a646e09e90f735d8bdee70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4395ee3783a646e09e90f735d8bdee70, State: Initialized, Role: FOLLOWER
I20260811 12:02:42.465312 13947 consensus_queue.cc:260] T 0917901cd1074e90a8d3bb266d540e30 P 4395ee3783a646e09e90f735d8bdee70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } }
I20260811 12:02:42.465726 13959 tablet_replica.cc:439] T 0917901cd1074e90a8d3bb266d540e30 P 4395ee3783a646e09e90f735d8bdee70: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } } }
I20260811 12:02:42.465912 13947 ts_tablet_manager.cc:1428] T 0917901cd1074e90a8d3bb266d540e30 P 4395ee3783a646e09e90f735d8bdee70: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 12:02:42.466022 13959 tablet_replica.cc:442] T 0917901cd1074e90a8d3bb266d540e30 P 4395ee3783a646e09e90f735d8bdee70: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:02:42.584684 13961 raft_consensus.cc:482] T 0917901cd1074e90a8d3bb266d540e30 P 8967cd6438d942b89dedf51a53d1f081 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:02:42.584937 13961 raft_consensus.cc:504] T 0917901cd1074e90a8d3bb266d540e30 P 8967cd6438d942b89dedf51a53d1f081 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } }
I20260811 12:02:42.586550 13961 leader_election.cc:288] T 0917901cd1074e90a8d3bb266d540e30 P 8967cd6438d942b89dedf51a53d1f081 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e286e4695ccd4ff48b17493f78c4629d (127.9.221.129:37553), 4395ee3783a646e09e90f735d8bdee70 (127.9.221.130:38365)
I20260811 12:02:42.594985 13937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:42.586950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:44919 (local address 127.9.221.130:38365)
0811 12:02:42.587235 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:42.587244 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:42.587884 (+ 640us) server_negotiation.cc:407] Connection header received
0811 12:02:42.587967 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:42.587974 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:42.588046 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:42.588136 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:02:42.588687 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.589779 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.592524 (+ 2745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.593750 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.593925 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:42.594001 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:42.594579 (+ 578us) server_negotiation.cc:299] Negotiation successful
0811 12:02:42.594712 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260811 12:02:42.595013 13941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:42.586936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:45107 (local address 127.9.221.129:37553)
0811 12:02:42.587430 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:42.587436 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:42.587521 (+ 85us) server_negotiation.cc:407] Connection header received
0811 12:02:42.587787 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:42.587791 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:42.587880 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:42.587990 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:02:42.588503 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.589642 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.592524 (+ 2882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.593750 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.593918 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:42.594001 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:42.594579 (+ 578us) server_negotiation.cc:299] Negotiation successful
0811 12:02:42.594712 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337}
I20260811 12:02:42.595111 13985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:42.586720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:37553 (local address 127.9.221.131:45107)
0811 12:02:42.587391 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:42.587418 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:42.587612 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:42.588019 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:42.588022 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:42.588042 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:02:42.588325 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:42.588336 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.589812 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.589815 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:42.592359 (+ 2544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:42.592374 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.593929 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.593936 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:42.594299 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:42.594436 (+ 137us) client_negotiation.cc:769] Sending connection context
0811 12:02:42.594536 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 12:02:42.594678 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"mutex_wait_us":311,"thread_start_us":428,"threads_started":1}
I20260811 12:02:42.595333 13986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:42.586828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.130:38365 (local address 127.9.221.131:44919)
0811 12:02:42.587763 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:42.587789 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:42.587878 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:42.588165 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:42.588169 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:42.588187 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:02:42.588526 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:42.588543 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.589900 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.589906 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:42.592359 (+ 2453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:42.592374 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.593986 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.593993 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:42.594297 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:42.594441 (+ 144us) client_negotiation.cc:769] Sending connection context
0811 12:02:42.594536 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 12:02:42.594679 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":821,"mutex_wait_us":260,"thread_start_us":632,"threads_started":1}
I20260811 12:02:42.595860 13344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0917901cd1074e90a8d3bb266d540e30" candidate_uuid: "8967cd6438d942b89dedf51a53d1f081" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e286e4695ccd4ff48b17493f78c4629d" is_pre_election: true
I20260811 12:02:42.596172 13344 raft_consensus.cc:2454] T 0917901cd1074e90a8d3bb266d540e30 P e286e4695ccd4ff48b17493f78c4629d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8967cd6438d942b89dedf51a53d1f081 in term 0.
I20260811 12:02:42.596473 13585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0917901cd1074e90a8d3bb266d540e30" candidate_uuid: "8967cd6438d942b89dedf51a53d1f081" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4395ee3783a646e09e90f735d8bdee70" is_pre_election: true
I20260811 12:02:42.596659 13695 leader_election.cc:302] T 0917901cd1074e90a8d3bb266d540e30 P 8967cd6438d942b89dedf51a53d1f081 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8967cd6438d942b89dedf51a53d1f081, e286e4695ccd4ff48b17493f78c4629d; no voters:
I20260811 12:02:42.596920 13585 raft_consensus.cc:2454] T 0917901cd1074e90a8d3bb266d540e30 P 4395ee3783a646e09e90f735d8bdee70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8967cd6438d942b89dedf51a53d1f081 in term 0.
I20260811 12:02:42.597074 13961 raft_consensus.cc:2781] T 0917901cd1074e90a8d3bb266d540e30 P 8967cd6438d942b89dedf51a53d1f081 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:02:42.597198 13961 raft_consensus.cc:482] T 0917901cd1074e90a8d3bb266d540e30 P 8967cd6438d942b89dedf51a53d1f081 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:02:42.597321 13961 raft_consensus.cc:3037] T 0917901cd1074e90a8d3bb266d540e30 P 8967cd6438d942b89dedf51a53d1f081 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:02:42.599627 13961 raft_consensus.cc:504] T 0917901cd1074e90a8d3bb266d540e30 P 8967cd6438d942b89dedf51a53d1f081 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } }
I20260811 12:02:42.600345 13961 leader_election.cc:288] T 0917901cd1074e90a8d3bb266d540e30 P 8967cd6438d942b89dedf51a53d1f081 [CANDIDATE]: Term 1 election: Requested vote from peers e286e4695ccd4ff48b17493f78c4629d (127.9.221.129:37553), 4395ee3783a646e09e90f735d8bdee70 (127.9.221.130:38365)
I20260811 12:02:42.600695 13344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0917901cd1074e90a8d3bb266d540e30" candidate_uuid: "8967cd6438d942b89dedf51a53d1f081" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e286e4695ccd4ff48b17493f78c4629d"
I20260811 12:02:42.600870 13585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0917901cd1074e90a8d3bb266d540e30" candidate_uuid: "8967cd6438d942b89dedf51a53d1f081" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4395ee3783a646e09e90f735d8bdee70"
I20260811 12:02:42.600924 13344 raft_consensus.cc:3037] T 0917901cd1074e90a8d3bb266d540e30 P e286e4695ccd4ff48b17493f78c4629d [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:02:42.601081 13585 raft_consensus.cc:3037] T 0917901cd1074e90a8d3bb266d540e30 P 4395ee3783a646e09e90f735d8bdee70 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:02:42.602991 13344 raft_consensus.cc:2454] T 0917901cd1074e90a8d3bb266d540e30 P e286e4695ccd4ff48b17493f78c4629d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8967cd6438d942b89dedf51a53d1f081 in term 1.
I20260811 12:02:42.603238 13585 raft_consensus.cc:2454] T 0917901cd1074e90a8d3bb266d540e30 P 4395ee3783a646e09e90f735d8bdee70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8967cd6438d942b89dedf51a53d1f081 in term 1.
I20260811 12:02:42.603390 13695 leader_election.cc:302] T 0917901cd1074e90a8d3bb266d540e30 P 8967cd6438d942b89dedf51a53d1f081 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8967cd6438d942b89dedf51a53d1f081, e286e4695ccd4ff48b17493f78c4629d; no voters:
I20260811 12:02:42.603727 13961 raft_consensus.cc:2781] T 0917901cd1074e90a8d3bb266d540e30 P 8967cd6438d942b89dedf51a53d1f081 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:02:42.604096 13961 raft_consensus.cc:686] T 0917901cd1074e90a8d3bb266d540e30 P 8967cd6438d942b89dedf51a53d1f081 [term 1 LEADER]: Becoming Leader. State: Replica: 8967cd6438d942b89dedf51a53d1f081, State: Running, Role: LEADER
I20260811 12:02:42.604564 13961 consensus_queue.cc:237] T 0917901cd1074e90a8d3bb266d540e30 P 8967cd6438d942b89dedf51a53d1f081 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } }
I20260811 12:02:42.608264 13987 tablet_replica.cc:439] T 0917901cd1074e90a8d3bb266d540e30 P 8967cd6438d942b89dedf51a53d1f081: TxnStatusTablet state changed. Reason: New leader 8967cd6438d942b89dedf51a53d1f081. Latest consensus state: current_term: 1 leader_uuid: "8967cd6438d942b89dedf51a53d1f081" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } } }
I20260811 12:02:42.608883 13987 tablet_replica.cc:442] T 0917901cd1074e90a8d3bb266d540e30 P 8967cd6438d942b89dedf51a53d1f081: This TxnStatusTablet replica's current role is: LEADER
I20260811 12:02:42.609593 13989 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 12:02:42.609467 13071 catalog_manager.cc:5495] T 0917901cd1074e90a8d3bb266d540e30 P 8967cd6438d942b89dedf51a53d1f081 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8967cd6438d942b89dedf51a53d1f081 (127.9.221.131). New cstate: current_term: 1 leader_uuid: "8967cd6438d942b89dedf51a53d1f081" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } health_report { overall_health: HEALTHY } } }
I20260811 12:02:42.609856 13989 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 12:02:42.657924 13946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:42.651817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45452 (local address 127.9.221.131:41221)
0811 12:02:42.651981 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:42.651985 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:42.652002 (+ 17us) server_negotiation.cc:407] Connection header received
0811 12:02:42.652202 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:42.652206 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:42.652297 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:42.652406 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:02:42.652858 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.653701 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.655608 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.656066 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.656159 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:42.656676 (+ 517us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:02:42.657248 (+ 572us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:02:42.657458 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:42.657518 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 12:02:42.657595 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 12:02:42.657949 13976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:42.651492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.131:41221 (local address 127.0.0.1:45452)
0811 12:02:42.651822 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:42.651844 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:42.651984 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:42.652482 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:42.652488 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:42.652501 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 12:02:42.652713 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:42.652724 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.653835 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.653841 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:42.655474 (+ 1633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:42.655491 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.656228 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.656235 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:42.656427 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:42.656511 (+ 84us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 12:02:42.657402 (+ 891us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 12:02:42.657423 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 12:02:42.657503 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 12:02:42.657644 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 12:02:42.665303 13585 raft_consensus.cc:1261] T 0917901cd1074e90a8d3bb266d540e30 P 4395ee3783a646e09e90f735d8bdee70 [term 1 FOLLOWER]: Refusing update from remote peer 8967cd6438d942b89dedf51a53d1f081: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:02:42.665426 13344 raft_consensus.cc:1261] T 0917901cd1074e90a8d3bb266d540e30 P e286e4695ccd4ff48b17493f78c4629d [term 1 FOLLOWER]: Refusing update from remote peer 8967cd6438d942b89dedf51a53d1f081: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:02:42.666265 13987 consensus_queue.cc:1035] T 0917901cd1074e90a8d3bb266d540e30 P 8967cd6438d942b89dedf51a53d1f081 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:02:42.666620 13961 consensus_queue.cc:1035] T 0917901cd1074e90a8d3bb266d540e30 P 8967cd6438d942b89dedf51a53d1f081 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:02:42.682633 13960 tablet_replica.cc:439] T 0917901cd1074e90a8d3bb266d540e30 P e286e4695ccd4ff48b17493f78c4629d: TxnStatusTablet state changed. Reason: New leader 8967cd6438d942b89dedf51a53d1f081. Latest consensus state: current_term: 1 leader_uuid: "8967cd6438d942b89dedf51a53d1f081" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } } }
I20260811 12:02:42.682950 13960 tablet_replica.cc:442] T 0917901cd1074e90a8d3bb266d540e30 P e286e4695ccd4ff48b17493f78c4629d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:02:42.683112 13959 tablet_replica.cc:439] T 0917901cd1074e90a8d3bb266d540e30 P 4395ee3783a646e09e90f735d8bdee70: TxnStatusTablet state changed. Reason: New leader 8967cd6438d942b89dedf51a53d1f081. Latest consensus state: current_term: 1 leader_uuid: "8967cd6438d942b89dedf51a53d1f081" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } } }
I20260811 12:02:42.683458 13959 tablet_replica.cc:442] T 0917901cd1074e90a8d3bb266d540e30 P 4395ee3783a646e09e90f735d8bdee70: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:02:42.686065 13994 mvcc.cc:205] Tried to move back new op lower bound from 7317298227867451392 to 7317298227632914432. Current Snapshot: MvccSnapshot[applied={T|T < 7317298227867451392}]
I20260811 12:02:42.687150 13987 tablet_replica.cc:439] T 0917901cd1074e90a8d3bb266d540e30 P 8967cd6438d942b89dedf51a53d1f081: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8967cd6438d942b89dedf51a53d1f081" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } } }
I20260811 12:02:42.687417 13987 tablet_replica.cc:442] T 0917901cd1074e90a8d3bb266d540e30 P 8967cd6438d942b89dedf51a53d1f081: This TxnStatusTablet replica's current role is: LEADER
I20260811 12:02:42.687682 13987 tablet_replica.cc:439] T 0917901cd1074e90a8d3bb266d540e30 P 8967cd6438d942b89dedf51a53d1f081: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8967cd6438d942b89dedf51a53d1f081" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } } }
I20260811 12:02:42.687923 13987 tablet_replica.cc:442] T 0917901cd1074e90a8d3bb266d540e30 P 8967cd6438d942b89dedf51a53d1f081: This TxnStatusTablet replica's current role is: LEADER
I20260811 12:02:42.690264 13998 mvcc.cc:205] Tried to move back new op lower bound from 7317298227867451392 to 7317298227632914432. Current Snapshot: MvccSnapshot[applied={T|T < 7317298227867451392}]
I20260811 12:02:42.694248 13960 tablet_replica.cc:439] T 0917901cd1074e90a8d3bb266d540e30 P e286e4695ccd4ff48b17493f78c4629d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8967cd6438d942b89dedf51a53d1f081" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } } }
I20260811 12:02:42.694489 13960 tablet_replica.cc:442] T 0917901cd1074e90a8d3bb266d540e30 P e286e4695ccd4ff48b17493f78c4629d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:02:42.706126 13959 tablet_replica.cc:439] T 0917901cd1074e90a8d3bb266d540e30 P 4395ee3783a646e09e90f735d8bdee70: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8967cd6438d942b89dedf51a53d1f081" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } } }
I20260811 12:02:42.706528 13959 tablet_replica.cc:442] T 0917901cd1074e90a8d3bb266d540e30 P 4395ee3783a646e09e90f735d8bdee70: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:02:42.716130 13937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:42.705079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58048 (local address 127.9.221.130:38365)
0811 12:02:42.705256 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:42.705262 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:42.710113 (+ 4851us) server_negotiation.cc:407] Connection header received
0811 12:02:42.710325 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:42.710328 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:42.710406 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:42.710530 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:02:42.711032 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.711834 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.713878 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.714293 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.714389 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:42.714895 (+ 506us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:02:42.715464 (+ 569us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:02:42.715565 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:42.715758 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 12:02:42.715823 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 12:02:42.732561 13946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:42.726431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:37423 (local address 127.9.221.131:41221)
0811 12:02:42.726675 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:42.726682 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:42.727052 (+ 370us) server_negotiation.cc:407] Connection header received
0811 12:02:42.727267 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:42.727270 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:42.727336 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:42.727410 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:02:42.728039 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.728783 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.730825 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.731252 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.731349 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:42.731937 (+ 588us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:02:42.732039 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:02:42.732113 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:42.732284 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 12:02:42.732344 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260811 12:02:42.732791 14023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:42.726228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.131:41221 (local address 127.9.221.130:37423)
0811 12:02:42.726938 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:42.726959 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:42.727113 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:42.727457 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:42.727462 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:42.727475 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 12:02:42.727872 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:42.727885 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.728918 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.728925 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:42.730682 (+ 1757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:42.730697 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.731429 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.731437 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:42.731659 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:42.731738 (+ 79us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 12:02:42.732160 (+ 422us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 12:02:42.732172 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 12:02:42.732258 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 12:02:42.732398 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":384,"threads_started":1}
I20260811 12:02:42.743111 13344 raft_consensus.cc:1261] T 43e05a60bf144ca4aae527ab1a290f72 P e286e4695ccd4ff48b17493f78c4629d [term 1 FOLLOWER]: Refusing update from remote peer 4395ee3783a646e09e90f735d8bdee70: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:02:42.743448 13823 raft_consensus.cc:1261] T 43e05a60bf144ca4aae527ab1a290f72 P 8967cd6438d942b89dedf51a53d1f081 [term 1 FOLLOWER]: Refusing update from remote peer 4395ee3783a646e09e90f735d8bdee70: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:02:42.743805 13964 consensus_queue.cc:1035] T 43e05a60bf144ca4aae527ab1a290f72 P 4395ee3783a646e09e90f735d8bdee70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:02:42.744181 13959 consensus_queue.cc:1035] T 43e05a60bf144ca4aae527ab1a290f72 P 4395ee3783a646e09e90f735d8bdee70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:02:42.748900 13996 mvcc.cc:205] Tried to move back new op lower bound from 7317298228189884416 to 7317298226652991488. Current Snapshot: MvccSnapshot[applied={T|T < 7317298228189884416}]
I20260811 12:02:42.778607 14037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:42.771364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.130:38365 (local address 127.9.221.131:48447)
0811 12:02:42.772130 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:42.772151 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:42.772302 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:42.772844 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:42.772851 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:42.772865 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 12:02:42.773272 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:42.773283 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.774552 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.774558 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:42.776505 (+ 1947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:42.776520 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.777406 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.777415 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:42.777603 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:42.777682 (+ 79us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 12:02:42.778124 (+ 442us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 12:02:42.778133 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 12:02:42.778205 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 12:02:42.778309 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":401,"threads_started":1}
I20260811 12:02:42.778601 13937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:42.771353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:48447 (local address 127.9.221.130:38365)
0811 12:02:42.771577 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:42.771584 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:42.772241 (+ 657us) server_negotiation.cc:407] Connection header received
0811 12:02:42.772476 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:42.772484 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:42.772567 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:42.772627 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:02:42.773446 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.774415 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.776633 (+ 2218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.777215 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.777358 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:42.777887 (+ 529us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:02:42.778018 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:02:42.778109 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:42.778229 (+ 120us) server_negotiation.cc:299] Negotiation successful
0811 12:02:42.778314 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 12:02:42.819252 13069 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39420:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:02:42.819538 13069 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39420:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:02:42.823566 13069 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d924d872636d40318a5089eb6fccbf6c: Sending DeleteTablet for 3 replicas of tablet 43e05a60bf144ca4aae527ab1a290f72
I20260811 12:02:42.824921 13310 tablet_service.cc:1514] Processing DeleteTablet for tablet 43e05a60bf144ca4aae527ab1a290f72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:02:42 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:43408
I20260811 12:02:42.825033 13538 tablet_service.cc:1514] Processing DeleteTablet for tablet 43e05a60bf144ca4aae527ab1a290f72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:02:42 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:58036
I20260811 12:02:42.825661 13781 tablet_service.cc:1514] Processing DeleteTablet for tablet 43e05a60bf144ca4aae527ab1a290f72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:02:42 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:45438
I20260811 12:02:42.831892 13069 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39420:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:02:42.832895 13069 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:02:42.840402 14048 tablet_replica.cc:330] T 43e05a60bf144ca4aae527ab1a290f72 P 8967cd6438d942b89dedf51a53d1f081: stopping tablet replica
I20260811 12:02:42.841553 13537 tablet_service.cc:1467] Processing CreateTablet for tablet 59c9775c715643ddaa373a08218bb9fe (DEFAULT_TABLE table=test-table [id=c93ac09b78a4490ca0bbe702f561c948]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:02:42.842120 13537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59c9775c715643ddaa373a08218bb9fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:02:42.844244 13309 tablet_service.cc:1467] Processing CreateTablet for tablet 59c9775c715643ddaa373a08218bb9fe (DEFAULT_TABLE table=test-table [id=c93ac09b78a4490ca0bbe702f561c948]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:02:42.844892 13309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59c9775c715643ddaa373a08218bb9fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:02:42.847517 14046 tablet_replica.cc:330] T 43e05a60bf144ca4aae527ab1a290f72 P e286e4695ccd4ff48b17493f78c4629d: stopping tablet replica
I20260811 12:02:42.848133 14046 raft_consensus.cc:2229] T 43e05a60bf144ca4aae527ab1a290f72 P e286e4695ccd4ff48b17493f78c4629d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:02:42.848500 14046 raft_consensus.cc:2258] T 43e05a60bf144ca4aae527ab1a290f72 P e286e4695ccd4ff48b17493f78c4629d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:02:42.850548 14046 ts_tablet_manager.cc:1905] T 43e05a60bf144ca4aae527ab1a290f72 P e286e4695ccd4ff48b17493f78c4629d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:02:42.855768 13947 tablet_bootstrap.cc:492] T 59c9775c715643ddaa373a08218bb9fe P 4395ee3783a646e09e90f735d8bdee70: Bootstrap starting.
I20260811 12:02:42.857470 13947 tablet_bootstrap.cc:654] T 59c9775c715643ddaa373a08218bb9fe P 4395ee3783a646e09e90f735d8bdee70: Neither blocks nor log segments found. Creating new log.
I20260811 12:02:42.858500 14046 ts_tablet_manager.cc:1918] T 43e05a60bf144ca4aae527ab1a290f72 P e286e4695ccd4ff48b17493f78c4629d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:02:42.858655 14046 log.cc:1196] T 43e05a60bf144ca4aae527ab1a290f72 P e286e4695ccd4ff48b17493f78c4629d: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-0/wal/wals/43e05a60bf144ca4aae527ab1a290f72
I20260811 12:02:42.859216 14046 ts_tablet_manager.cc:1939] T 43e05a60bf144ca4aae527ab1a290f72 P e286e4695ccd4ff48b17493f78c4629d: Deleting consensus metadata
I20260811 12:02:42.861145 13048 catalog_manager.cc:4841] TS e286e4695ccd4ff48b17493f78c4629d (127.9.221.129:37553): tablet 43e05a60bf144ca4aae527ab1a290f72 (table test-table [id=b0bfa997d98943269a7403e22346ba33]) successfully deleted
I20260811 12:02:42.841266 13779 tablet_service.cc:1467] Processing CreateTablet for tablet 59c9775c715643ddaa373a08218bb9fe (DEFAULT_TABLE table=test-table [id=c93ac09b78a4490ca0bbe702f561c948]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:02:42.862501 14048 raft_consensus.cc:2229] T 43e05a60bf144ca4aae527ab1a290f72 P 8967cd6438d942b89dedf51a53d1f081 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:02:42.862586 13779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59c9775c715643ddaa373a08218bb9fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:02:42.862926 14048 raft_consensus.cc:2258] T 43e05a60bf144ca4aae527ab1a290f72 P 8967cd6438d942b89dedf51a53d1f081 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:02:42.864979 13949 tablet_bootstrap.cc:492] T 59c9775c715643ddaa373a08218bb9fe P e286e4695ccd4ff48b17493f78c4629d: Bootstrap starting.
I20260811 12:02:42.865430 14047 tablet_replica.cc:330] T 43e05a60bf144ca4aae527ab1a290f72 P 4395ee3783a646e09e90f735d8bdee70: stopping tablet replica
I20260811 12:02:42.865815 14047 raft_consensus.cc:2229] T 43e05a60bf144ca4aae527ab1a290f72 P 4395ee3783a646e09e90f735d8bdee70 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:02:42.866338 14047 raft_consensus.cc:2258] T 43e05a60bf144ca4aae527ab1a290f72 P 4395ee3783a646e09e90f735d8bdee70 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:02:42.866595 13949 tablet_bootstrap.cc:654] T 59c9775c715643ddaa373a08218bb9fe P e286e4695ccd4ff48b17493f78c4629d: Neither blocks nor log segments found. Creating new log.
I20260811 12:02:42.868707 14047 ts_tablet_manager.cc:1905] T 43e05a60bf144ca4aae527ab1a290f72 P 4395ee3783a646e09e90f735d8bdee70: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:02:42.870800 13957 tablet_bootstrap.cc:492] T 59c9775c715643ddaa373a08218bb9fe P 8967cd6438d942b89dedf51a53d1f081: Bootstrap starting.
I20260811 12:02:42.872797 13957 tablet_bootstrap.cc:654] T 59c9775c715643ddaa373a08218bb9fe P 8967cd6438d942b89dedf51a53d1f081: Neither blocks nor log segments found. Creating new log.
I20260811 12:02:42.876407 14047 ts_tablet_manager.cc:1918] T 43e05a60bf144ca4aae527ab1a290f72 P 4395ee3783a646e09e90f735d8bdee70: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:02:42.876698 14047 log.cc:1196] T 43e05a60bf144ca4aae527ab1a290f72 P 4395ee3783a646e09e90f735d8bdee70: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-1/wal/wals/43e05a60bf144ca4aae527ab1a290f72
I20260811 12:02:42.877313 14047 ts_tablet_manager.cc:1939] T 43e05a60bf144ca4aae527ab1a290f72 P 4395ee3783a646e09e90f735d8bdee70: Deleting consensus metadata
I20260811 12:02:42.878907 13052 catalog_manager.cc:4841] TS 4395ee3783a646e09e90f735d8bdee70 (127.9.221.130:38365): tablet 43e05a60bf144ca4aae527ab1a290f72 (table test-table [id=b0bfa997d98943269a7403e22346ba33]) successfully deleted
I20260811 12:02:42.880326 13949 tablet_bootstrap.cc:492] T 59c9775c715643ddaa373a08218bb9fe P e286e4695ccd4ff48b17493f78c4629d: No bootstrap required, opened a new log
I20260811 12:02:42.880559 13949 ts_tablet_manager.cc:1397] T 59c9775c715643ddaa373a08218bb9fe P e286e4695ccd4ff48b17493f78c4629d: Time spent bootstrapping tablet: real 0.016s user 0.011s sys 0.000s
I20260811 12:02:42.881489 13949 raft_consensus.cc:348] T 59c9775c715643ddaa373a08218bb9fe P e286e4695ccd4ff48b17493f78c4629d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } }
I20260811 12:02:42.881808 13949 raft_consensus.cc:374] T 59c9775c715643ddaa373a08218bb9fe P e286e4695ccd4ff48b17493f78c4629d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:02:42.881899 13949 raft_consensus.cc:729] T 59c9775c715643ddaa373a08218bb9fe P e286e4695ccd4ff48b17493f78c4629d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e286e4695ccd4ff48b17493f78c4629d, State: Initialized, Role: FOLLOWER
I20260811 12:02:42.883230 13957 tablet_bootstrap.cc:492] T 59c9775c715643ddaa373a08218bb9fe P 8967cd6438d942b89dedf51a53d1f081: No bootstrap required, opened a new log
I20260811 12:02:42.883419 13957 ts_tablet_manager.cc:1397] T 59c9775c715643ddaa373a08218bb9fe P 8967cd6438d942b89dedf51a53d1f081: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.004s
I20260811 12:02:42.882282 13949 consensus_queue.cc:260] T 59c9775c715643ddaa373a08218bb9fe P e286e4695ccd4ff48b17493f78c4629d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } }
I20260811 12:02:42.884284 13957 raft_consensus.cc:348] T 59c9775c715643ddaa373a08218bb9fe P 8967cd6438d942b89dedf51a53d1f081 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } }
I20260811 12:02:42.884575 13957 raft_consensus.cc:374] T 59c9775c715643ddaa373a08218bb9fe P 8967cd6438d942b89dedf51a53d1f081 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:02:42.884693 13957 raft_consensus.cc:729] T 59c9775c715643ddaa373a08218bb9fe P 8967cd6438d942b89dedf51a53d1f081 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8967cd6438d942b89dedf51a53d1f081, State: Initialized, Role: FOLLOWER
I20260811 12:02:42.884977 13957 consensus_queue.cc:260] T 59c9775c715643ddaa373a08218bb9fe P 8967cd6438d942b89dedf51a53d1f081 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } }
I20260811 12:02:42.887753 13949 ts_tablet_manager.cc:1428] T 59c9775c715643ddaa373a08218bb9fe P e286e4695ccd4ff48b17493f78c4629d: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260811 12:02:42.887917 13957 ts_tablet_manager.cc:1428] T 59c9775c715643ddaa373a08218bb9fe P 8967cd6438d942b89dedf51a53d1f081: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260811 12:02:42.888700 14048 ts_tablet_manager.cc:1905] T 43e05a60bf144ca4aae527ab1a290f72 P 8967cd6438d942b89dedf51a53d1f081: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:02:42.891985 13947 tablet_bootstrap.cc:492] T 59c9775c715643ddaa373a08218bb9fe P 4395ee3783a646e09e90f735d8bdee70: No bootstrap required, opened a new log
I20260811 12:02:42.892189 13947 ts_tablet_manager.cc:1397] T 59c9775c715643ddaa373a08218bb9fe P 4395ee3783a646e09e90f735d8bdee70: Time spent bootstrapping tablet: real 0.036s user 0.003s sys 0.007s
I20260811 12:02:42.893095 13947 raft_consensus.cc:348] T 59c9775c715643ddaa373a08218bb9fe P 4395ee3783a646e09e90f735d8bdee70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } }
I20260811 12:02:42.893375 13947 raft_consensus.cc:374] T 59c9775c715643ddaa373a08218bb9fe P 4395ee3783a646e09e90f735d8bdee70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:02:42.893465 13947 raft_consensus.cc:729] T 59c9775c715643ddaa373a08218bb9fe P 4395ee3783a646e09e90f735d8bdee70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4395ee3783a646e09e90f735d8bdee70, State: Initialized, Role: FOLLOWER
I20260811 12:02:42.893752 13947 consensus_queue.cc:260] T 59c9775c715643ddaa373a08218bb9fe P 4395ee3783a646e09e90f735d8bdee70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } }
I20260811 12:02:42.895282 13947 ts_tablet_manager.cc:1428] T 59c9775c715643ddaa373a08218bb9fe P 4395ee3783a646e09e90f735d8bdee70: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 12:02:42.897660 14048 ts_tablet_manager.cc:1918] T 43e05a60bf144ca4aae527ab1a290f72 P 8967cd6438d942b89dedf51a53d1f081: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:02:42.897913 14048 log.cc:1196] T 43e05a60bf144ca4aae527ab1a290f72 P 8967cd6438d942b89dedf51a53d1f081: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-2/wal/wals/43e05a60bf144ca4aae527ab1a290f72
I20260811 12:02:42.898574 14048 ts_tablet_manager.cc:1939] T 43e05a60bf144ca4aae527ab1a290f72 P 8967cd6438d942b89dedf51a53d1f081: Deleting consensus metadata
I20260811 12:02:42.900473 13052 catalog_manager.cc:4841] TS 8967cd6438d942b89dedf51a53d1f081 (127.9.221.131:41221): tablet 43e05a60bf144ca4aae527ab1a290f72 (table test-table [id=b0bfa997d98943269a7403e22346ba33]) successfully deleted
I20260811 12:02:42.904215 14001 raft_consensus.cc:482] T 59c9775c715643ddaa373a08218bb9fe P 8967cd6438d942b89dedf51a53d1f081 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:02:42.904404 14001 raft_consensus.cc:504] T 59c9775c715643ddaa373a08218bb9fe P 8967cd6438d942b89dedf51a53d1f081 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } }
I20260811 12:02:42.905136 14001 leader_election.cc:288] T 59c9775c715643ddaa373a08218bb9fe P 8967cd6438d942b89dedf51a53d1f081 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e286e4695ccd4ff48b17493f78c4629d (127.9.221.129:37553), 4395ee3783a646e09e90f735d8bdee70 (127.9.221.130:38365)
I20260811 12:02:42.905580 13343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59c9775c715643ddaa373a08218bb9fe" candidate_uuid: "8967cd6438d942b89dedf51a53d1f081" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e286e4695ccd4ff48b17493f78c4629d" is_pre_election: true
I20260811 12:02:42.905689 13585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59c9775c715643ddaa373a08218bb9fe" candidate_uuid: "8967cd6438d942b89dedf51a53d1f081" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4395ee3783a646e09e90f735d8bdee70" is_pre_election: true
I20260811 12:02:42.905885 13343 raft_consensus.cc:2454] T 59c9775c715643ddaa373a08218bb9fe P e286e4695ccd4ff48b17493f78c4629d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8967cd6438d942b89dedf51a53d1f081 in term 0.
I20260811 12:02:42.905951 13585 raft_consensus.cc:2454] T 59c9775c715643ddaa373a08218bb9fe P 4395ee3783a646e09e90f735d8bdee70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8967cd6438d942b89dedf51a53d1f081 in term 0.
I20260811 12:02:42.906424 13699 leader_election.cc:302] T 59c9775c715643ddaa373a08218bb9fe P 8967cd6438d942b89dedf51a53d1f081 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4395ee3783a646e09e90f735d8bdee70, 8967cd6438d942b89dedf51a53d1f081; no voters:
I20260811 12:02:42.906884 14001 raft_consensus.cc:2781] T 59c9775c715643ddaa373a08218bb9fe P 8967cd6438d942b89dedf51a53d1f081 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:02:42.907014 14001 raft_consensus.cc:482] T 59c9775c715643ddaa373a08218bb9fe P 8967cd6438d942b89dedf51a53d1f081 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:02:42.907125 14001 raft_consensus.cc:3037] T 59c9775c715643ddaa373a08218bb9fe P 8967cd6438d942b89dedf51a53d1f081 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:02:42.910379 14001 raft_consensus.cc:504] T 59c9775c715643ddaa373a08218bb9fe P 8967cd6438d942b89dedf51a53d1f081 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } }
I20260811 12:02:42.911412 13343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59c9775c715643ddaa373a08218bb9fe" candidate_uuid: "8967cd6438d942b89dedf51a53d1f081" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e286e4695ccd4ff48b17493f78c4629d"
I20260811 12:02:42.911422 13585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59c9775c715643ddaa373a08218bb9fe" candidate_uuid: "8967cd6438d942b89dedf51a53d1f081" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4395ee3783a646e09e90f735d8bdee70"
I20260811 12:02:42.911614 13585 raft_consensus.cc:3037] T 59c9775c715643ddaa373a08218bb9fe P 4395ee3783a646e09e90f735d8bdee70 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:02:42.911610 13343 raft_consensus.cc:3037] T 59c9775c715643ddaa373a08218bb9fe P e286e4695ccd4ff48b17493f78c4629d [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:02:42.914820 13585 raft_consensus.cc:2454] T 59c9775c715643ddaa373a08218bb9fe P 4395ee3783a646e09e90f735d8bdee70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8967cd6438d942b89dedf51a53d1f081 in term 1.
I20260811 12:02:42.915227 13343 raft_consensus.cc:2454] T 59c9775c715643ddaa373a08218bb9fe P e286e4695ccd4ff48b17493f78c4629d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8967cd6438d942b89dedf51a53d1f081 in term 1.
I20260811 12:02:42.915246 13699 leader_election.cc:302] T 59c9775c715643ddaa373a08218bb9fe P 8967cd6438d942b89dedf51a53d1f081 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4395ee3783a646e09e90f735d8bdee70, 8967cd6438d942b89dedf51a53d1f081; no voters:
I20260811 12:02:42.915544 14001 leader_election.cc:288] T 59c9775c715643ddaa373a08218bb9fe P 8967cd6438d942b89dedf51a53d1f081 [CANDIDATE]: Term 1 election: Requested vote from peers e286e4695ccd4ff48b17493f78c4629d (127.9.221.129:37553), 4395ee3783a646e09e90f735d8bdee70 (127.9.221.130:38365)
I20260811 12:02:42.915673 13961 raft_consensus.cc:2781] T 59c9775c715643ddaa373a08218bb9fe P 8967cd6438d942b89dedf51a53d1f081 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:02:42.915824 13961 raft_consensus.cc:686] T 59c9775c715643ddaa373a08218bb9fe P 8967cd6438d942b89dedf51a53d1f081 [term 1 LEADER]: Becoming Leader. State: Replica: 8967cd6438d942b89dedf51a53d1f081, State: Running, Role: LEADER
I20260811 12:02:42.916127 13961 consensus_queue.cc:237] T 59c9775c715643ddaa373a08218bb9fe P 8967cd6438d942b89dedf51a53d1f081 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } }
I20260811 12:02:42.919044 13071 catalog_manager.cc:5495] T 59c9775c715643ddaa373a08218bb9fe P 8967cd6438d942b89dedf51a53d1f081 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8967cd6438d942b89dedf51a53d1f081 (127.9.221.131). New cstate: current_term: 1 leader_uuid: "8967cd6438d942b89dedf51a53d1f081" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } health_report { overall_health: HEALTHY } } }
I20260811 12:02:42.948154 13946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:42.941070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45468 (local address 127.9.221.131:41221)
0811 12:02:42.941639 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:42.941646 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:42.941662 (+ 16us) server_negotiation.cc:407] Connection header received
0811 12:02:42.941728 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:42.941734 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:42.941814 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:42.941913 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:02:42.942372 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.943560 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.945617 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.946443 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.946603 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:42.947058 (+ 455us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:02:42.947218 (+ 160us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:02:42.947308 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:42.947520 (+ 212us) server_negotiation.cc:299] Negotiation successful
0811 12:02:42.947606 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422}
I20260811 12:02:42.969769 14037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:42.958793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.131:41221 (local address 127.9.221.131:41111)
0811 12:02:42.958985 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:42.959004 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:42.959124 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:42.964019 (+ 4895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:42.964025 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:42.964037 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 12:02:42.964212 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:42.964226 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.965752 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.965758 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:42.967436 (+ 1678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:42.967453 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.968567 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.968573 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:42.968731 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:42.968781 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 12:02:42.969204 (+ 423us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 12:02:42.969212 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 12:02:42.969279 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 12:02:42.969428 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 12:02:42.970438 13946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:42.959856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:41111 (local address 127.9.221.131:41221)
0811 12:02:42.963532 (+ 3676us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:42.963538 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:42.963559 (+ 21us) server_negotiation.cc:407] Connection header received
0811 12:02:42.963624 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:42.963631 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:42.963743 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:42.963841 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:02:42.964353 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.965565 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.967596 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:42.968298 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:42.968419 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:42.968957 (+ 538us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:02:42.969081 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:02:42.969165 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:42.970107 (+ 942us) server_negotiation.cc:299] Negotiation successful
0811 12:02:42.970188 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3535}
I20260811 12:02:42.985785 13585 raft_consensus.cc:1261] T 59c9775c715643ddaa373a08218bb9fe P 4395ee3783a646e09e90f735d8bdee70 [term 1 FOLLOWER]: Refusing update from remote peer 8967cd6438d942b89dedf51a53d1f081: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:02:42.986366 13987 consensus_queue.cc:1035] T 59c9775c715643ddaa373a08218bb9fe P 8967cd6438d942b89dedf51a53d1f081 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:02:42.987411 13343 raft_consensus.cc:1261] T 59c9775c715643ddaa373a08218bb9fe P e286e4695ccd4ff48b17493f78c4629d [term 1 FOLLOWER]: Refusing update from remote peer 8967cd6438d942b89dedf51a53d1f081: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:02:42.988009 13987 consensus_queue.cc:1035] T 59c9775c715643ddaa373a08218bb9fe P 8967cd6438d942b89dedf51a53d1f081 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 12:02:43.025655 13994 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 12:02:44.647419 13071 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39420:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:02:44.647728 13071 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39420:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:02:44.651932 13071 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d924d872636d40318a5089eb6fccbf6c: Sending DeleteTablet for 3 replicas of tablet 59c9775c715643ddaa373a08218bb9fe
I20260811 12:02:44.652731 13309 tablet_service.cc:1514] Processing DeleteTablet for tablet 59c9775c715643ddaa373a08218bb9fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:02:44 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:43408
I20260811 12:02:44.653000 13538 tablet_service.cc:1514] Processing DeleteTablet for tablet 59c9775c715643ddaa373a08218bb9fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:02:44 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:58036
I20260811 12:02:44.653120 13781 tablet_service.cc:1514] Processing DeleteTablet for tablet 59c9775c715643ddaa373a08218bb9fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:02:44 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:45438
I20260811 12:02:44.668426 14099 tablet_replica.cc:330] T 59c9775c715643ddaa373a08218bb9fe P 4395ee3783a646e09e90f735d8bdee70: stopping tablet replica
I20260811 12:02:44.668704 14099 raft_consensus.cc:2229] T 59c9775c715643ddaa373a08218bb9fe P 4395ee3783a646e09e90f735d8bdee70 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:02:44.668977 14099 raft_consensus.cc:2258] T 59c9775c715643ddaa373a08218bb9fe P 4395ee3783a646e09e90f735d8bdee70 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:02:44.669565 14097 tablet_replica.cc:330] T 59c9775c715643ddaa373a08218bb9fe P e286e4695ccd4ff48b17493f78c4629d: stopping tablet replica
I20260811 12:02:44.669799 14097 raft_consensus.cc:2229] T 59c9775c715643ddaa373a08218bb9fe P e286e4695ccd4ff48b17493f78c4629d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:02:44.670048 14097 raft_consensus.cc:2258] T 59c9775c715643ddaa373a08218bb9fe P e286e4695ccd4ff48b17493f78c4629d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:02:44.671000 14099 ts_tablet_manager.cc:1905] T 59c9775c715643ddaa373a08218bb9fe P 4395ee3783a646e09e90f735d8bdee70: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:02:44.675846 14097 ts_tablet_manager.cc:1905] T 59c9775c715643ddaa373a08218bb9fe P e286e4695ccd4ff48b17493f78c4629d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:02:44.676203 14100 tablet_replica.cc:330] T 59c9775c715643ddaa373a08218bb9fe P 8967cd6438d942b89dedf51a53d1f081: stopping tablet replica
I20260811 12:02:44.676440 14100 raft_consensus.cc:2229] T 59c9775c715643ddaa373a08218bb9fe P 8967cd6438d942b89dedf51a53d1f081 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:02:44.677153 14100 raft_consensus.cc:2258] T 59c9775c715643ddaa373a08218bb9fe P 8967cd6438d942b89dedf51a53d1f081 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:02:44.678112 14100 ts_tablet_manager.cc:1905] T 59c9775c715643ddaa373a08218bb9fe P 8967cd6438d942b89dedf51a53d1f081: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:02:44.678725 14099 ts_tablet_manager.cc:1918] T 59c9775c715643ddaa373a08218bb9fe P 4395ee3783a646e09e90f735d8bdee70: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 12:02:44.678860 14099 log.cc:1196] T 59c9775c715643ddaa373a08218bb9fe P 4395ee3783a646e09e90f735d8bdee70: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-1/wal/wals/59c9775c715643ddaa373a08218bb9fe
I20260811 12:02:44.679368 14099 ts_tablet_manager.cc:1939] T 59c9775c715643ddaa373a08218bb9fe P 4395ee3783a646e09e90f735d8bdee70: Deleting consensus metadata
I20260811 12:02:44.680619 13052 catalog_manager.cc:4841] TS 4395ee3783a646e09e90f735d8bdee70 (127.9.221.130:38365): tablet 59c9775c715643ddaa373a08218bb9fe (table test-table [id=c93ac09b78a4490ca0bbe702f561c948]) successfully deleted
I20260811 12:02:44.681545 14097 ts_tablet_manager.cc:1918] T 59c9775c715643ddaa373a08218bb9fe P e286e4695ccd4ff48b17493f78c4629d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 12:02:44.681691 14097 log.cc:1196] T 59c9775c715643ddaa373a08218bb9fe P e286e4695ccd4ff48b17493f78c4629d: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-0/wal/wals/59c9775c715643ddaa373a08218bb9fe
I20260811 12:02:44.682063 14097 ts_tablet_manager.cc:1939] T 59c9775c715643ddaa373a08218bb9fe P e286e4695ccd4ff48b17493f78c4629d: Deleting consensus metadata
I20260811 12:02:44.682976 13048 catalog_manager.cc:4841] TS e286e4695ccd4ff48b17493f78c4629d (127.9.221.129:37553): tablet 59c9775c715643ddaa373a08218bb9fe (table test-table [id=c93ac09b78a4490ca0bbe702f561c948]) successfully deleted
I20260811 12:02:44.683755 14100 ts_tablet_manager.cc:1918] T 59c9775c715643ddaa373a08218bb9fe P 8967cd6438d942b89dedf51a53d1f081: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 12:02:44.683879 14100 log.cc:1196] T 59c9775c715643ddaa373a08218bb9fe P 8967cd6438d942b89dedf51a53d1f081: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-2/wal/wals/59c9775c715643ddaa373a08218bb9fe
I20260811 12:02:44.684392 14100 ts_tablet_manager.cc:1939] T 59c9775c715643ddaa373a08218bb9fe P 8967cd6438d942b89dedf51a53d1f081: Deleting consensus metadata
I20260811 12:02:44.685272 13052 catalog_manager.cc:4841] TS 8967cd6438d942b89dedf51a53d1f081 (127.9.221.131:41221): tablet 59c9775c715643ddaa373a08218bb9fe (table test-table [id=c93ac09b78a4490ca0bbe702f561c948]) successfully deleted
Aug 11 12:02:44 dist-test-slave-mjjr krb5kdc[12997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449764, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 12:02:44 dist-test-slave-mjjr krb5kdc[12997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449764, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:02:44.721534 14110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:44.707094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39442 (local address 127.9.221.190:41979)
0811 12:02:44.707766 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:44.707772 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:44.707798 (+ 26us) server_negotiation.cc:407] Connection header received
0811 12:02:44.708015 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:44.708019 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:44.708085 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:44.708200 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:44.709533 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:44.710319 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:44.711507 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:44.712015 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:44.712156 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:44.716138 (+ 3982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:44.716144 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:44.716148 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:44.716182 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:44.718825 (+ 2643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:44.719495 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:44.719500 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:44.719504 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:44.719633 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:44.720060 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:44.720067 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:44.720070 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:44.720345 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:44.720602 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:44.720905 (+ 303us) server_negotiation.cc:299] Negotiation successful
0811 12:02:44.721149 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":325,"threads_started":1}
I20260811 12:02:44.724977 13071 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39442:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:02:44.725536 13071 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:02:44.731935 13309 tablet_service.cc:1467] Processing CreateTablet for tablet e9fb2a92330d422da0f84d0411e8e8bf (DEFAULT_TABLE table=test-table [id=c6d5c4e7268c481f93232f0c353a244b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:02:44.732129 13538 tablet_service.cc:1467] Processing CreateTablet for tablet e9fb2a92330d422da0f84d0411e8e8bf (DEFAULT_TABLE table=test-table [id=c6d5c4e7268c481f93232f0c353a244b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:02:44.732446 13309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9fb2a92330d422da0f84d0411e8e8bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:02:44.732344 13781 tablet_service.cc:1467] Processing CreateTablet for tablet e9fb2a92330d422da0f84d0411e8e8bf (DEFAULT_TABLE table=test-table [id=c6d5c4e7268c481f93232f0c353a244b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:02:44.732579 13538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9fb2a92330d422da0f84d0411e8e8bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:02:44.732784 13781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9fb2a92330d422da0f84d0411e8e8bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:02:44.739388 14111 tablet_bootstrap.cc:492] T e9fb2a92330d422da0f84d0411e8e8bf P 8967cd6438d942b89dedf51a53d1f081: Bootstrap starting.
I20260811 12:02:44.740216 14116 tablet_bootstrap.cc:492] T e9fb2a92330d422da0f84d0411e8e8bf P 4395ee3783a646e09e90f735d8bdee70: Bootstrap starting.
I20260811 12:02:44.743108 14111 tablet_bootstrap.cc:654] T e9fb2a92330d422da0f84d0411e8e8bf P 8967cd6438d942b89dedf51a53d1f081: Neither blocks nor log segments found. Creating new log.
I20260811 12:02:44.743880 14116 tablet_bootstrap.cc:654] T e9fb2a92330d422da0f84d0411e8e8bf P 4395ee3783a646e09e90f735d8bdee70: Neither blocks nor log segments found. Creating new log.
I20260811 12:02:44.747268 14113 tablet_bootstrap.cc:492] T e9fb2a92330d422da0f84d0411e8e8bf P e286e4695ccd4ff48b17493f78c4629d: Bootstrap starting.
I20260811 12:02:44.749290 14111 tablet_bootstrap.cc:492] T e9fb2a92330d422da0f84d0411e8e8bf P 8967cd6438d942b89dedf51a53d1f081: No bootstrap required, opened a new log
I20260811 12:02:44.749490 14111 ts_tablet_manager.cc:1397] T e9fb2a92330d422da0f84d0411e8e8bf P 8967cd6438d942b89dedf51a53d1f081: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260811 12:02:44.750129 14116 tablet_bootstrap.cc:492] T e9fb2a92330d422da0f84d0411e8e8bf P 4395ee3783a646e09e90f735d8bdee70: No bootstrap required, opened a new log
I20260811 12:02:44.750365 14116 ts_tablet_manager.cc:1397] T e9fb2a92330d422da0f84d0411e8e8bf P 4395ee3783a646e09e90f735d8bdee70: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.008s
I20260811 12:02:44.750500 14113 tablet_bootstrap.cc:654] T e9fb2a92330d422da0f84d0411e8e8bf P e286e4695ccd4ff48b17493f78c4629d: Neither blocks nor log segments found. Creating new log.
I20260811 12:02:44.750705 14111 raft_consensus.cc:348] T e9fb2a92330d422da0f84d0411e8e8bf P 8967cd6438d942b89dedf51a53d1f081 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } }
I20260811 12:02:44.750983 14111 raft_consensus.cc:374] T e9fb2a92330d422da0f84d0411e8e8bf P 8967cd6438d942b89dedf51a53d1f081 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:02:44.751215 14111 raft_consensus.cc:729] T e9fb2a92330d422da0f84d0411e8e8bf P 8967cd6438d942b89dedf51a53d1f081 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8967cd6438d942b89dedf51a53d1f081, State: Initialized, Role: FOLLOWER
I20260811 12:02:44.751605 14111 consensus_queue.cc:260] T e9fb2a92330d422da0f84d0411e8e8bf P 8967cd6438d942b89dedf51a53d1f081 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } }
I20260811 12:02:44.752053 14116 raft_consensus.cc:348] T e9fb2a92330d422da0f84d0411e8e8bf P 4395ee3783a646e09e90f735d8bdee70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } }
I20260811 12:02:44.752444 14116 raft_consensus.cc:374] T e9fb2a92330d422da0f84d0411e8e8bf P 4395ee3783a646e09e90f735d8bdee70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:02:44.752619 14116 raft_consensus.cc:729] T e9fb2a92330d422da0f84d0411e8e8bf P 4395ee3783a646e09e90f735d8bdee70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4395ee3783a646e09e90f735d8bdee70, State: Initialized, Role: FOLLOWER
I20260811 12:02:44.753073 14111 ts_tablet_manager.cc:1428] T e9fb2a92330d422da0f84d0411e8e8bf P 8967cd6438d942b89dedf51a53d1f081: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260811 12:02:44.752966 14116 consensus_queue.cc:260] T e9fb2a92330d422da0f84d0411e8e8bf P 4395ee3783a646e09e90f735d8bdee70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } }
I20260811 12:02:44.754375 14116 ts_tablet_manager.cc:1428] T e9fb2a92330d422da0f84d0411e8e8bf P 4395ee3783a646e09e90f735d8bdee70: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20260811 12:02:44.757119 14113 tablet_bootstrap.cc:492] T e9fb2a92330d422da0f84d0411e8e8bf P e286e4695ccd4ff48b17493f78c4629d: No bootstrap required, opened a new log
I20260811 12:02:44.757366 14113 ts_tablet_manager.cc:1397] T e9fb2a92330d422da0f84d0411e8e8bf P e286e4695ccd4ff48b17493f78c4629d: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.007s
I20260811 12:02:44.758322 14113 raft_consensus.cc:348] T e9fb2a92330d422da0f84d0411e8e8bf P e286e4695ccd4ff48b17493f78c4629d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } }
I20260811 12:02:44.758641 14113 raft_consensus.cc:374] T e9fb2a92330d422da0f84d0411e8e8bf P e286e4695ccd4ff48b17493f78c4629d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:02:44.758795 14113 raft_consensus.cc:729] T e9fb2a92330d422da0f84d0411e8e8bf P e286e4695ccd4ff48b17493f78c4629d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e286e4695ccd4ff48b17493f78c4629d, State: Initialized, Role: FOLLOWER
I20260811 12:02:44.759076 14113 consensus_queue.cc:260] T e9fb2a92330d422da0f84d0411e8e8bf P e286e4695ccd4ff48b17493f78c4629d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } }
I20260811 12:02:44.760023 14113 ts_tablet_manager.cc:1428] T e9fb2a92330d422da0f84d0411e8e8bf P e286e4695ccd4ff48b17493f78c4629d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 12:02:44.865684 14127 raft_consensus.cc:482] T e9fb2a92330d422da0f84d0411e8e8bf P e286e4695ccd4ff48b17493f78c4629d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:02:44.865978 14127 raft_consensus.cc:504] T e9fb2a92330d422da0f84d0411e8e8bf P e286e4695ccd4ff48b17493f78c4629d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } }
I20260811 12:02:44.867540 14127 leader_election.cc:288] T e9fb2a92330d422da0f84d0411e8e8bf P e286e4695ccd4ff48b17493f78c4629d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4395ee3783a646e09e90f735d8bdee70 (127.9.221.130:38365), 8967cd6438d942b89dedf51a53d1f081 (127.9.221.131:41221)
I20260811 12:02:44.877353 14133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:44.867962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:37783 (local address 127.9.221.130:38365)
0811 12:02:44.868646 (+ 684us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:44.868650 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:44.869465 (+ 815us) server_negotiation.cc:407] Connection header received
0811 12:02:44.869558 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:44.869564 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:44.869648 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:44.869773 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:02:44.870316 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:44.871937 (+ 1621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:44.874727 (+ 2790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:44.876205 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:44.876433 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:44.876525 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:44.876844 (+ 319us) server_negotiation.cc:299] Negotiation successful
0811 12:02:44.876982 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":408,"threads_started":1}
I20260811 12:02:44.877394 14134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:44.867919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.130:38365 (local address 127.9.221.129:37783)
0811 12:02:44.868777 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:44.868807 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:44.868962 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:44.869803 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:44.869807 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:44.869831 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:02:44.870125 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:44.870134 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:44.872067 (+ 1933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:44.872070 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:44.874562 (+ 2492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:44.874577 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:44.876420 (+ 1843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:44.876428 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:44.876625 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:44.876667 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 12:02:44.876816 (+ 149us) client_negotiation.cc:240] Negotiation successful
0811 12:02:44.876975 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":688,"mutex_wait_us":122,"thread_start_us":490,"threads_started":1}
I20260811 12:02:44.877628 14136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:44.869061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:37913 (local address 127.9.221.131:41221)
0811 12:02:44.870054 (+ 993us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:44.870059 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:44.870100 (+ 41us) server_negotiation.cc:407] Connection header received
0811 12:02:44.870156 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:44.870159 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:44.870243 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:44.870354 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:02:44.870906 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:44.872055 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:44.874718 (+ 2663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:44.876383 (+ 1665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:44.876644 (+ 261us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:44.876731 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:44.877071 (+ 340us) server_negotiation.cc:299] Negotiation successful
0811 12:02:44.877255 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":725,"thread_start_us":468,"threads_started":1}
I20260811 12:02:44.877667 14135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:44.868876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.131:41221 (local address 127.9.221.129:37913)
0811 12:02:44.869630 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:44.869650 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:44.869798 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:44.870380 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:44.870383 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:44.870405 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:02:44.870739 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:44.870764 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:44.872187 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:44.872191 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:44.874562 (+ 2371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:44.874577 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:44.876731 (+ 2154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:44.876739 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:44.876903 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:44.876934 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 12:02:44.877045 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 12:02:44.877188 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"mutex_wait_us":51,"thread_start_us":341,"threads_started":1}
I20260811 12:02:44.878463 13823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9fb2a92330d422da0f84d0411e8e8bf" candidate_uuid: "e286e4695ccd4ff48b17493f78c4629d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8967cd6438d942b89dedf51a53d1f081" is_pre_election: true
I20260811 12:02:44.878448 13583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9fb2a92330d422da0f84d0411e8e8bf" candidate_uuid: "e286e4695ccd4ff48b17493f78c4629d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4395ee3783a646e09e90f735d8bdee70" is_pre_election: true
I20260811 12:02:44.878748 13583 raft_consensus.cc:2454] T e9fb2a92330d422da0f84d0411e8e8bf P 4395ee3783a646e09e90f735d8bdee70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e286e4695ccd4ff48b17493f78c4629d in term 0.
I20260811 12:02:44.878747 13823 raft_consensus.cc:2454] T e9fb2a92330d422da0f84d0411e8e8bf P 8967cd6438d942b89dedf51a53d1f081 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e286e4695ccd4ff48b17493f78c4629d in term 0.
I20260811 12:02:44.879244 13241 leader_election.cc:302] T e9fb2a92330d422da0f84d0411e8e8bf P e286e4695ccd4ff48b17493f78c4629d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4395ee3783a646e09e90f735d8bdee70, e286e4695ccd4ff48b17493f78c4629d; no voters:
I20260811 12:02:44.879602 14127 raft_consensus.cc:2781] T e9fb2a92330d422da0f84d0411e8e8bf P e286e4695ccd4ff48b17493f78c4629d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:02:44.879758 14127 raft_consensus.cc:482] T e9fb2a92330d422da0f84d0411e8e8bf P e286e4695ccd4ff48b17493f78c4629d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:02:44.879884 14127 raft_consensus.cc:3037] T e9fb2a92330d422da0f84d0411e8e8bf P e286e4695ccd4ff48b17493f78c4629d [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:02:44.882067 14127 raft_consensus.cc:504] T e9fb2a92330d422da0f84d0411e8e8bf P e286e4695ccd4ff48b17493f78c4629d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } }
I20260811 12:02:44.882706 14127 leader_election.cc:288] T e9fb2a92330d422da0f84d0411e8e8bf P e286e4695ccd4ff48b17493f78c4629d [CANDIDATE]: Term 1 election: Requested vote from peers 4395ee3783a646e09e90f735d8bdee70 (127.9.221.130:38365), 8967cd6438d942b89dedf51a53d1f081 (127.9.221.131:41221)
I20260811 12:02:44.883129 13583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9fb2a92330d422da0f84d0411e8e8bf" candidate_uuid: "e286e4695ccd4ff48b17493f78c4629d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4395ee3783a646e09e90f735d8bdee70"
I20260811 12:02:44.883219 13823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9fb2a92330d422da0f84d0411e8e8bf" candidate_uuid: "e286e4695ccd4ff48b17493f78c4629d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8967cd6438d942b89dedf51a53d1f081"
I20260811 12:02:44.883315 13583 raft_consensus.cc:3037] T e9fb2a92330d422da0f84d0411e8e8bf P 4395ee3783a646e09e90f735d8bdee70 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:02:44.883476 13823 raft_consensus.cc:3037] T e9fb2a92330d422da0f84d0411e8e8bf P 8967cd6438d942b89dedf51a53d1f081 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:02:44.885346 13583 raft_consensus.cc:2454] T e9fb2a92330d422da0f84d0411e8e8bf P 4395ee3783a646e09e90f735d8bdee70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e286e4695ccd4ff48b17493f78c4629d in term 1.
I20260811 12:02:44.885514 13823 raft_consensus.cc:2454] T e9fb2a92330d422da0f84d0411e8e8bf P 8967cd6438d942b89dedf51a53d1f081 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e286e4695ccd4ff48b17493f78c4629d in term 1.
I20260811 12:02:44.885859 13241 leader_election.cc:302] T e9fb2a92330d422da0f84d0411e8e8bf P e286e4695ccd4ff48b17493f78c4629d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4395ee3783a646e09e90f735d8bdee70, e286e4695ccd4ff48b17493f78c4629d; no voters:
I20260811 12:02:44.886160 14127 raft_consensus.cc:2781] T e9fb2a92330d422da0f84d0411e8e8bf P e286e4695ccd4ff48b17493f78c4629d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:02:44.886715 14127 raft_consensus.cc:686] T e9fb2a92330d422da0f84d0411e8e8bf P e286e4695ccd4ff48b17493f78c4629d [term 1 LEADER]: Becoming Leader. State: Replica: e286e4695ccd4ff48b17493f78c4629d, State: Running, Role: LEADER
I20260811 12:02:44.887135 14127 consensus_queue.cc:237] T e9fb2a92330d422da0f84d0411e8e8bf P e286e4695ccd4ff48b17493f78c4629d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } }
I20260811 12:02:44.890596 13069 catalog_manager.cc:5495] T e9fb2a92330d422da0f84d0411e8e8bf P e286e4695ccd4ff48b17493f78c4629d reported cstate change: term changed from 0 to 1, leader changed from <none> to e286e4695ccd4ff48b17493f78c4629d (127.9.221.129). New cstate: current_term: 1 leader_uuid: "e286e4695ccd4ff48b17493f78c4629d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } health_report { overall_health: UNKNOWN } } }
I20260811 12:02:44.942857 14142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:44.936868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43410 (local address 127.9.221.129:37553)
0811 12:02:44.937502 (+ 634us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:44.937507 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:44.937526 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:02:44.937608 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:44.937612 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:44.937690 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:44.937833 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:02:44.938222 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:44.939278 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:44.940625 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:44.941045 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:44.941141 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:44.941548 (+ 407us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:02:44.942179 (+ 631us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:02:44.942319 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:44.942511 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 12:02:44.942588 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":397,"threads_started":1}
I20260811 12:02:44.964702 14136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:44.954695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:58309 (local address 127.9.221.131:41221)
0811 12:02:44.954876 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:44.954881 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:44.956078 (+ 1197us) server_negotiation.cc:407] Connection header received
0811 12:02:44.956304 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:44.956308 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:44.956378 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:44.956454 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:02:44.957310 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:44.958681 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:44.961624 (+ 2943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:44.962625 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:44.962771 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:44.963681 (+ 910us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:02:44.963890 (+ 209us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:02:44.964016 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:44.964163 (+ 147us) server_negotiation.cc:299] Negotiation successful
0811 12:02:44.964258 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 12:02:44.965214 14148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:44.954632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.131:41221 (local address 127.9.221.129:58309)
0811 12:02:44.955945 (+ 1313us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:44.955979 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:44.956171 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:44.956491 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:44.956494 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:44.956503 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 12:02:44.957108 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:44.957120 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:44.958828 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:44.958834 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:44.961445 (+ 2611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:44.961468 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:44.962883 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:44.962890 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:44.963257 (+ 367us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:44.963329 (+ 72us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 12:02:44.964021 (+ 692us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 12:02:44.964033 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 12:02:44.964117 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 12:02:44.964258 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1066,"thread_start_us":787,"threads_started":1}
I20260811 12:02:44.977999 13823 raft_consensus.cc:1261] T e9fb2a92330d422da0f84d0411e8e8bf P 8967cd6438d942b89dedf51a53d1f081 [term 1 FOLLOWER]: Refusing update from remote peer e286e4695ccd4ff48b17493f78c4629d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:02:44.978190 13583 raft_consensus.cc:1261] T e9fb2a92330d422da0f84d0411e8e8bf P 4395ee3783a646e09e90f735d8bdee70 [term 1 FOLLOWER]: Refusing update from remote peer e286e4695ccd4ff48b17493f78c4629d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:02:44.978992 14137 consensus_queue.cc:1035] T e9fb2a92330d422da0f84d0411e8e8bf P e286e4695ccd4ff48b17493f78c4629d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:02:44.980008 14127 consensus_queue.cc:1035] T e9fb2a92330d422da0f84d0411e8e8bf P e286e4695ccd4ff48b17493f78c4629d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:02:45.034198 14142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:45.024317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:50583 (local address 127.9.221.129:37553)
0811 12:02:45.024575 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:45.024582 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:45.026276 (+ 1694us) server_negotiation.cc:407] Connection header received
0811 12:02:45.026586 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:45.026591 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:45.026670 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:45.026754 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:02:45.027602 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:45.028947 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:45.031708 (+ 2761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:45.032580 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:45.032702 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:45.033320 (+ 618us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:02:45.033444 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:02:45.033562 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:45.033671 (+ 109us) server_negotiation.cc:299] Negotiation successful
0811 12:02:45.033750 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125}
I20260811 12:02:45.034415 14157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:45.024686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:37553 (local address 127.9.221.131:50583)
0811 12:02:45.026159 (+ 1473us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:45.026180 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:45.026328 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:45.026787 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:45.026790 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:45.026800 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 12:02:45.027411 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:45.027421 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:45.029118 (+ 1697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:45.029124 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:45.031559 (+ 2435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:45.031572 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:45.032890 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:45.032897 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:45.033092 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:45.033131 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 12:02:45.033542 (+ 411us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 12:02:45.033552 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 12:02:45.033641 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 12:02:45.033781 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1248,"thread_start_us":718,"threads_started":1}
I20260811 12:02:45.074734 13069 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:02:45.075125 13069 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:02:45.079196 13069 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d924d872636d40318a5089eb6fccbf6c: Sending DeleteTablet for 3 replicas of tablet e9fb2a92330d422da0f84d0411e8e8bf
I20260811 12:02:45.080684 13538 tablet_service.cc:1514] Processing DeleteTablet for tablet e9fb2a92330d422da0f84d0411e8e8bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:02:45 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:58036
I20260811 12:02:45.080652 13781 tablet_service.cc:1514] Processing DeleteTablet for tablet e9fb2a92330d422da0f84d0411e8e8bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:02:45 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:45438
I20260811 12:02:45.081125 14100 tablet_replica.cc:330] T e9fb2a92330d422da0f84d0411e8e8bf P 8967cd6438d942b89dedf51a53d1f081: stopping tablet replica
I20260811 12:02:45.081089 14099 tablet_replica.cc:330] T e9fb2a92330d422da0f84d0411e8e8bf P 4395ee3783a646e09e90f735d8bdee70: stopping tablet replica
I20260811 12:02:45.081436 14100 raft_consensus.cc:2229] T e9fb2a92330d422da0f84d0411e8e8bf P 8967cd6438d942b89dedf51a53d1f081 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:02:45.081460 14099 raft_consensus.cc:2229] T e9fb2a92330d422da0f84d0411e8e8bf P 4395ee3783a646e09e90f735d8bdee70 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:02:45.081712 14100 raft_consensus.cc:2258] T e9fb2a92330d422da0f84d0411e8e8bf P 8967cd6438d942b89dedf51a53d1f081 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:02:45.081756 14099 raft_consensus.cc:2258] T e9fb2a92330d422da0f84d0411e8e8bf P 4395ee3783a646e09e90f735d8bdee70 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:02:45.081707 13069 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39442:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:02:45.082659 13069 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:02:45.082625 13309 tablet_service.cc:1514] Processing DeleteTablet for tablet e9fb2a92330d422da0f84d0411e8e8bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:02:45 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:43408
I20260811 12:02:45.082921 14097 tablet_replica.cc:330] T e9fb2a92330d422da0f84d0411e8e8bf P e286e4695ccd4ff48b17493f78c4629d: stopping tablet replica
I20260811 12:02:45.083146 14097 raft_consensus.cc:2229] T e9fb2a92330d422da0f84d0411e8e8bf P e286e4695ccd4ff48b17493f78c4629d [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:02:45.083909 14097 raft_consensus.cc:2258] T e9fb2a92330d422da0f84d0411e8e8bf P e286e4695ccd4ff48b17493f78c4629d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:02:45.084090 14100 ts_tablet_manager.cc:1905] T e9fb2a92330d422da0f84d0411e8e8bf P 8967cd6438d942b89dedf51a53d1f081: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:02:45.089053 14099 ts_tablet_manager.cc:1905] T e9fb2a92330d422da0f84d0411e8e8bf P 4395ee3783a646e09e90f735d8bdee70: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:02:45.091346 13538 tablet_service.cc:1467] Processing CreateTablet for tablet 012f6ff7436c4098b3dcde5b687e0c44 (DEFAULT_TABLE table=test-table [id=7d2f42576e3441af80be864edbf26cb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:02:45.091414 13309 tablet_service.cc:1467] Processing CreateTablet for tablet 012f6ff7436c4098b3dcde5b687e0c44 (DEFAULT_TABLE table=test-table [id=7d2f42576e3441af80be864edbf26cb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:02:45.092365 13309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 012f6ff7436c4098b3dcde5b687e0c44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:02:45.093931 14097 ts_tablet_manager.cc:1905] T e9fb2a92330d422da0f84d0411e8e8bf P e286e4695ccd4ff48b17493f78c4629d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:02:45.095160 13781 tablet_service.cc:1467] Processing CreateTablet for tablet 012f6ff7436c4098b3dcde5b687e0c44 (DEFAULT_TABLE table=test-table [id=7d2f42576e3441af80be864edbf26cb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:02:45.095973 13538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 012f6ff7436c4098b3dcde5b687e0c44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:02:45.103818 13781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 012f6ff7436c4098b3dcde5b687e0c44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:02:45.103878 14113 tablet_bootstrap.cc:492] T 012f6ff7436c4098b3dcde5b687e0c44 P e286e4695ccd4ff48b17493f78c4629d: Bootstrap starting.
I20260811 12:02:45.108052 14113 tablet_bootstrap.cc:654] T 012f6ff7436c4098b3dcde5b687e0c44 P e286e4695ccd4ff48b17493f78c4629d: Neither blocks nor log segments found. Creating new log.
I20260811 12:02:45.109874 14097 ts_tablet_manager.cc:1918] T e9fb2a92330d422da0f84d0411e8e8bf P e286e4695ccd4ff48b17493f78c4629d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:02:45.110220 14097 log.cc:1196] T e9fb2a92330d422da0f84d0411e8e8bf P e286e4695ccd4ff48b17493f78c4629d: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-0/wal/wals/e9fb2a92330d422da0f84d0411e8e8bf
I20260811 12:02:45.111122 14097 ts_tablet_manager.cc:1939] T e9fb2a92330d422da0f84d0411e8e8bf P e286e4695ccd4ff48b17493f78c4629d: Deleting consensus metadata
I20260811 12:02:45.112538 13048 catalog_manager.cc:4841] TS e286e4695ccd4ff48b17493f78c4629d (127.9.221.129:37553): tablet e9fb2a92330d422da0f84d0411e8e8bf (table test-table [id=c6d5c4e7268c481f93232f0c353a244b]) successfully deleted
I20260811 12:02:45.114993 14100 ts_tablet_manager.cc:1918] T e9fb2a92330d422da0f84d0411e8e8bf P 8967cd6438d942b89dedf51a53d1f081: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:02:45.115272 14100 log.cc:1196] T e9fb2a92330d422da0f84d0411e8e8bf P 8967cd6438d942b89dedf51a53d1f081: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-2/wal/wals/e9fb2a92330d422da0f84d0411e8e8bf
I20260811 12:02:45.120105 14099 ts_tablet_manager.cc:1918] T e9fb2a92330d422da0f84d0411e8e8bf P 4395ee3783a646e09e90f735d8bdee70: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:02:45.120445 14099 log.cc:1196] T e9fb2a92330d422da0f84d0411e8e8bf P 4395ee3783a646e09e90f735d8bdee70: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-1/wal/wals/e9fb2a92330d422da0f84d0411e8e8bf
I20260811 12:02:45.121209 14099 ts_tablet_manager.cc:1939] T e9fb2a92330d422da0f84d0411e8e8bf P 4395ee3783a646e09e90f735d8bdee70: Deleting consensus metadata
I20260811 12:02:45.122097 14113 tablet_bootstrap.cc:492] T 012f6ff7436c4098b3dcde5b687e0c44 P e286e4695ccd4ff48b17493f78c4629d: No bootstrap required, opened a new log
I20260811 12:02:45.122391 14113 ts_tablet_manager.cc:1397] T 012f6ff7436c4098b3dcde5b687e0c44 P e286e4695ccd4ff48b17493f78c4629d: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.003s
I20260811 12:02:45.122478 13052 catalog_manager.cc:4841] TS 4395ee3783a646e09e90f735d8bdee70 (127.9.221.130:38365): tablet e9fb2a92330d422da0f84d0411e8e8bf (table test-table [id=c6d5c4e7268c481f93232f0c353a244b]) successfully deleted
I20260811 12:02:45.123781 14113 raft_consensus.cc:348] T 012f6ff7436c4098b3dcde5b687e0c44 P e286e4695ccd4ff48b17493f78c4629d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } }
I20260811 12:02:45.124116 14100 ts_tablet_manager.cc:1939] T e9fb2a92330d422da0f84d0411e8e8bf P 8967cd6438d942b89dedf51a53d1f081: Deleting consensus metadata
I20260811 12:02:45.124228 14113 raft_consensus.cc:374] T 012f6ff7436c4098b3dcde5b687e0c44 P e286e4695ccd4ff48b17493f78c4629d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:02:45.124454 14113 raft_consensus.cc:729] T 012f6ff7436c4098b3dcde5b687e0c44 P e286e4695ccd4ff48b17493f78c4629d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e286e4695ccd4ff48b17493f78c4629d, State: Initialized, Role: FOLLOWER
I20260811 12:02:45.124778 14116 tablet_bootstrap.cc:492] T 012f6ff7436c4098b3dcde5b687e0c44 P 4395ee3783a646e09e90f735d8bdee70: Bootstrap starting.
I20260811 12:02:45.124976 14113 consensus_queue.cc:260] T 012f6ff7436c4098b3dcde5b687e0c44 P e286e4695ccd4ff48b17493f78c4629d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } }
I20260811 12:02:45.125902 14113 ts_tablet_manager.cc:1428] T 012f6ff7436c4098b3dcde5b687e0c44 P e286e4695ccd4ff48b17493f78c4629d: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260811 12:02:45.127813 14116 tablet_bootstrap.cc:654] T 012f6ff7436c4098b3dcde5b687e0c44 P 4395ee3783a646e09e90f735d8bdee70: Neither blocks nor log segments found. Creating new log.
I20260811 12:02:45.128840 14111 tablet_bootstrap.cc:492] T 012f6ff7436c4098b3dcde5b687e0c44 P 8967cd6438d942b89dedf51a53d1f081: Bootstrap starting.
I20260811 12:02:45.129818 13052 catalog_manager.cc:4841] TS 8967cd6438d942b89dedf51a53d1f081 (127.9.221.131:41221): tablet e9fb2a92330d422da0f84d0411e8e8bf (table test-table [id=c6d5c4e7268c481f93232f0c353a244b]) successfully deleted
I20260811 12:02:45.131582 14111 tablet_bootstrap.cc:654] T 012f6ff7436c4098b3dcde5b687e0c44 P 8967cd6438d942b89dedf51a53d1f081: Neither blocks nor log segments found. Creating new log.
I20260811 12:02:45.132107 14116 tablet_bootstrap.cc:492] T 012f6ff7436c4098b3dcde5b687e0c44 P 4395ee3783a646e09e90f735d8bdee70: No bootstrap required, opened a new log
I20260811 12:02:45.132513 14116 ts_tablet_manager.cc:1397] T 012f6ff7436c4098b3dcde5b687e0c44 P 4395ee3783a646e09e90f735d8bdee70: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.001s
I20260811 12:02:45.133960 14116 raft_consensus.cc:348] T 012f6ff7436c4098b3dcde5b687e0c44 P 4395ee3783a646e09e90f735d8bdee70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } }
I20260811 12:02:45.134456 14116 raft_consensus.cc:374] T 012f6ff7436c4098b3dcde5b687e0c44 P 4395ee3783a646e09e90f735d8bdee70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:02:45.134647 14116 raft_consensus.cc:729] T 012f6ff7436c4098b3dcde5b687e0c44 P 4395ee3783a646e09e90f735d8bdee70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4395ee3783a646e09e90f735d8bdee70, State: Initialized, Role: FOLLOWER
I20260811 12:02:45.135013 14116 consensus_queue.cc:260] T 012f6ff7436c4098b3dcde5b687e0c44 P 4395ee3783a646e09e90f735d8bdee70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } }
I20260811 12:02:45.136552 14116 ts_tablet_manager.cc:1428] T 012f6ff7436c4098b3dcde5b687e0c44 P 4395ee3783a646e09e90f735d8bdee70: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260811 12:02:45.136564 14111 tablet_bootstrap.cc:492] T 012f6ff7436c4098b3dcde5b687e0c44 P 8967cd6438d942b89dedf51a53d1f081: No bootstrap required, opened a new log
I20260811 12:02:45.136873 14111 ts_tablet_manager.cc:1397] T 012f6ff7436c4098b3dcde5b687e0c44 P 8967cd6438d942b89dedf51a53d1f081: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.001s
I20260811 12:02:45.138072 14111 raft_consensus.cc:348] T 012f6ff7436c4098b3dcde5b687e0c44 P 8967cd6438d942b89dedf51a53d1f081 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } }
I20260811 12:02:45.138448 14111 raft_consensus.cc:374] T 012f6ff7436c4098b3dcde5b687e0c44 P 8967cd6438d942b89dedf51a53d1f081 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:02:45.138613 14111 raft_consensus.cc:729] T 012f6ff7436c4098b3dcde5b687e0c44 P 8967cd6438d942b89dedf51a53d1f081 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8967cd6438d942b89dedf51a53d1f081, State: Initialized, Role: FOLLOWER
I20260811 12:02:45.139002 14111 consensus_queue.cc:260] T 012f6ff7436c4098b3dcde5b687e0c44 P 8967cd6438d942b89dedf51a53d1f081 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } }
I20260811 12:02:45.140201 14111 ts_tablet_manager.cc:1428] T 012f6ff7436c4098b3dcde5b687e0c44 P 8967cd6438d942b89dedf51a53d1f081: Time spent starting tablet: real 0.003s user 0.002s sys 0.002s
I20260811 12:02:45.279556 14126 raft_consensus.cc:482] T 012f6ff7436c4098b3dcde5b687e0c44 P 4395ee3783a646e09e90f735d8bdee70 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:02:45.280035 14126 raft_consensus.cc:504] T 012f6ff7436c4098b3dcde5b687e0c44 P 4395ee3783a646e09e90f735d8bdee70 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } }
I20260811 12:02:45.281311 14126 leader_election.cc:288] T 012f6ff7436c4098b3dcde5b687e0c44 P 4395ee3783a646e09e90f735d8bdee70 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e286e4695ccd4ff48b17493f78c4629d (127.9.221.129:37553), 8967cd6438d942b89dedf51a53d1f081 (127.9.221.131:41221)
I20260811 12:02:45.281873 13344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "012f6ff7436c4098b3dcde5b687e0c44" candidate_uuid: "4395ee3783a646e09e90f735d8bdee70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e286e4695ccd4ff48b17493f78c4629d" is_pre_election: true
I20260811 12:02:45.281966 13823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "012f6ff7436c4098b3dcde5b687e0c44" candidate_uuid: "4395ee3783a646e09e90f735d8bdee70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8967cd6438d942b89dedf51a53d1f081" is_pre_election: true
I20260811 12:02:45.282224 13344 raft_consensus.cc:2454] T 012f6ff7436c4098b3dcde5b687e0c44 P e286e4695ccd4ff48b17493f78c4629d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4395ee3783a646e09e90f735d8bdee70 in term 0.
I20260811 12:02:45.282316 13823 raft_consensus.cc:2454] T 012f6ff7436c4098b3dcde5b687e0c44 P 8967cd6438d942b89dedf51a53d1f081 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4395ee3783a646e09e90f735d8bdee70 in term 0.
I20260811 12:02:45.282881 13462 leader_election.cc:302] T 012f6ff7436c4098b3dcde5b687e0c44 P 4395ee3783a646e09e90f735d8bdee70 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4395ee3783a646e09e90f735d8bdee70, e286e4695ccd4ff48b17493f78c4629d; no voters:
I20260811 12:02:45.283416 14126 raft_consensus.cc:2781] T 012f6ff7436c4098b3dcde5b687e0c44 P 4395ee3783a646e09e90f735d8bdee70 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:02:45.283630 14126 raft_consensus.cc:482] T 012f6ff7436c4098b3dcde5b687e0c44 P 4395ee3783a646e09e90f735d8bdee70 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:02:45.283754 14126 raft_consensus.cc:3037] T 012f6ff7436c4098b3dcde5b687e0c44 P 4395ee3783a646e09e90f735d8bdee70 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:02:45.286435 14126 raft_consensus.cc:504] T 012f6ff7436c4098b3dcde5b687e0c44 P 4395ee3783a646e09e90f735d8bdee70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } }
I20260811 12:02:45.287051 14126 leader_election.cc:288] T 012f6ff7436c4098b3dcde5b687e0c44 P 4395ee3783a646e09e90f735d8bdee70 [CANDIDATE]: Term 1 election: Requested vote from peers e286e4695ccd4ff48b17493f78c4629d (127.9.221.129:37553), 8967cd6438d942b89dedf51a53d1f081 (127.9.221.131:41221)
I20260811 12:02:45.287837 13823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "012f6ff7436c4098b3dcde5b687e0c44" candidate_uuid: "4395ee3783a646e09e90f735d8bdee70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8967cd6438d942b89dedf51a53d1f081"
I20260811 12:02:45.288072 13344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "012f6ff7436c4098b3dcde5b687e0c44" candidate_uuid: "4395ee3783a646e09e90f735d8bdee70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e286e4695ccd4ff48b17493f78c4629d"
I20260811 12:02:45.288172 13823 raft_consensus.cc:3037] T 012f6ff7436c4098b3dcde5b687e0c44 P 8967cd6438d942b89dedf51a53d1f081 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:02:45.288302 13344 raft_consensus.cc:3037] T 012f6ff7436c4098b3dcde5b687e0c44 P e286e4695ccd4ff48b17493f78c4629d [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:02:45.292033 13823 raft_consensus.cc:2454] T 012f6ff7436c4098b3dcde5b687e0c44 P 8967cd6438d942b89dedf51a53d1f081 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4395ee3783a646e09e90f735d8bdee70 in term 1.
I20260811 12:02:45.292550 13344 raft_consensus.cc:2454] T 012f6ff7436c4098b3dcde5b687e0c44 P e286e4695ccd4ff48b17493f78c4629d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4395ee3783a646e09e90f735d8bdee70 in term 1.
I20260811 12:02:45.292616 13464 leader_election.cc:302] T 012f6ff7436c4098b3dcde5b687e0c44 P 4395ee3783a646e09e90f735d8bdee70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4395ee3783a646e09e90f735d8bdee70, 8967cd6438d942b89dedf51a53d1f081; no voters:
I20260811 12:02:45.292965 14126 raft_consensus.cc:2781] T 012f6ff7436c4098b3dcde5b687e0c44 P 4395ee3783a646e09e90f735d8bdee70 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:02:45.293859 14126 raft_consensus.cc:686] T 012f6ff7436c4098b3dcde5b687e0c44 P 4395ee3783a646e09e90f735d8bdee70 [term 1 LEADER]: Becoming Leader. State: Replica: 4395ee3783a646e09e90f735d8bdee70, State: Running, Role: LEADER
I20260811 12:02:45.294302 14126 consensus_queue.cc:237] T 012f6ff7436c4098b3dcde5b687e0c44 P 4395ee3783a646e09e90f735d8bdee70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } }
I20260811 12:02:45.297672 13069 catalog_manager.cc:5495] T 012f6ff7436c4098b3dcde5b687e0c44 P 4395ee3783a646e09e90f735d8bdee70 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4395ee3783a646e09e90f735d8bdee70 (127.9.221.130). New cstate: current_term: 1 leader_uuid: "4395ee3783a646e09e90f735d8bdee70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4395ee3783a646e09e90f735d8bdee70" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 } health_report { overall_health: UNKNOWN } } }
I20260811 12:02:45.362644 14133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:45.355631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58052 (local address 127.9.221.130:38365)
0811 12:02:45.355831 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:45.355835 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:45.356312 (+ 477us) server_negotiation.cc:407] Connection header received
0811 12:02:45.356447 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:45.356454 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:45.356563 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:45.356715 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:02:45.357153 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:45.358471 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:45.360130 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:45.360970 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:45.361094 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:45.361679 (+ 585us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:02:45.361849 (+ 170us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:02:45.362027 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:45.362099 (+ 72us) server_negotiation.cc:299] Negotiation successful
0811 12:02:45.362202 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 12:02:45.374310 13823 raft_consensus.cc:1261] T 012f6ff7436c4098b3dcde5b687e0c44 P 8967cd6438d942b89dedf51a53d1f081 [term 1 FOLLOWER]: Refusing update from remote peer 4395ee3783a646e09e90f735d8bdee70: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:02:45.374557 13344 raft_consensus.cc:1261] T 012f6ff7436c4098b3dcde5b687e0c44 P e286e4695ccd4ff48b17493f78c4629d [term 1 FOLLOWER]: Refusing update from remote peer 4395ee3783a646e09e90f735d8bdee70: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:02:45.375145 14162 consensus_queue.cc:1035] T 012f6ff7436c4098b3dcde5b687e0c44 P 4395ee3783a646e09e90f735d8bdee70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8967cd6438d942b89dedf51a53d1f081" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260811 12:02:45.376456 14126 consensus_queue.cc:1035] T 012f6ff7436c4098b3dcde5b687e0c44 P 4395ee3783a646e09e90f735d8bdee70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e286e4695ccd4ff48b17493f78c4629d" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 12:02:45.401832 14069 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 12:02:47.038702 13071 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:02:47.038990 13071 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:02:47.045648 13071 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d924d872636d40318a5089eb6fccbf6c: Sending DeleteTablet for 3 replicas of tablet 012f6ff7436c4098b3dcde5b687e0c44
I20260811 12:02:47.046504 13309 tablet_service.cc:1514] Processing DeleteTablet for tablet 012f6ff7436c4098b3dcde5b687e0c44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:02:47 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:43408
I20260811 12:02:47.046835 13779 tablet_service.cc:1514] Processing DeleteTablet for tablet 012f6ff7436c4098b3dcde5b687e0c44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:02:47 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:45438
I20260811 12:02:47.047029 13538 tablet_service.cc:1514] Processing DeleteTablet for tablet 012f6ff7436c4098b3dcde5b687e0c44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:02:47 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:58036
I20260811 12:02:47.047874 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 13203
I20260811 12:02:47.059245 14214 tablet_replica.cc:330] T 012f6ff7436c4098b3dcde5b687e0c44 P 8967cd6438d942b89dedf51a53d1f081: stopping tablet replica
I20260811 12:02:47.059475 14214 raft_consensus.cc:2229] T 012f6ff7436c4098b3dcde5b687e0c44 P 8967cd6438d942b89dedf51a53d1f081 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:02:47.059756 14214 raft_consensus.cc:2258] T 012f6ff7436c4098b3dcde5b687e0c44 P 8967cd6438d942b89dedf51a53d1f081 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:02:47.061640 14214 ts_tablet_manager.cc:1905] T 012f6ff7436c4098b3dcde5b687e0c44 P 8967cd6438d942b89dedf51a53d1f081: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:02:47.064435 14215 tablet_replica.cc:330] T 012f6ff7436c4098b3dcde5b687e0c44 P 4395ee3783a646e09e90f735d8bdee70: stopping tablet replica
I20260811 12:02:47.064704 14215 raft_consensus.cc:2229] T 012f6ff7436c4098b3dcde5b687e0c44 P 4395ee3783a646e09e90f735d8bdee70 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:02:47.065177 14215 raft_consensus.cc:2258] T 012f6ff7436c4098b3dcde5b687e0c44 P 4395ee3783a646e09e90f735d8bdee70 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:02:47.066232 14215 ts_tablet_manager.cc:1905] T 012f6ff7436c4098b3dcde5b687e0c44 P 4395ee3783a646e09e90f735d8bdee70: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:02:47.070076 14214 ts_tablet_manager.cc:1918] T 012f6ff7436c4098b3dcde5b687e0c44 P 8967cd6438d942b89dedf51a53d1f081: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 12:02:47.070271 14214 log.cc:1196] T 012f6ff7436c4098b3dcde5b687e0c44 P 8967cd6438d942b89dedf51a53d1f081: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-2/wal/wals/012f6ff7436c4098b3dcde5b687e0c44
I20260811 12:02:47.070909 14214 ts_tablet_manager.cc:1939] T 012f6ff7436c4098b3dcde5b687e0c44 P 8967cd6438d942b89dedf51a53d1f081: Deleting consensus metadata
I20260811 12:02:47.072285 13052 catalog_manager.cc:4841] TS 8967cd6438d942b89dedf51a53d1f081 (127.9.221.131:41221): tablet 012f6ff7436c4098b3dcde5b687e0c44 (table test-table [id=7d2f42576e3441af80be864edbf26cb8]) successfully deleted
I20260811 12:02:47.074041 14215 ts_tablet_manager.cc:1918] T 012f6ff7436c4098b3dcde5b687e0c44 P 4395ee3783a646e09e90f735d8bdee70: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 12:02:47.074316 14215 log.cc:1196] T 012f6ff7436c4098b3dcde5b687e0c44 P 4395ee3783a646e09e90f735d8bdee70: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-1/wal/wals/012f6ff7436c4098b3dcde5b687e0c44
I20260811 12:02:47.074985 14215 ts_tablet_manager.cc:1939] T 012f6ff7436c4098b3dcde5b687e0c44 P 4395ee3783a646e09e90f735d8bdee70: Deleting consensus metadata
I20260811 12:02:47.076131 13052 catalog_manager.cc:4841] TS 4395ee3783a646e09e90f735d8bdee70 (127.9.221.130:38365): tablet 012f6ff7436c4098b3dcde5b687e0c44 (table test-table [id=7d2f42576e3441af80be864edbf26cb8]) successfully deleted
I20260811 12:02:47.083047 14211 tablet_replica.cc:330] T 012f6ff7436c4098b3dcde5b687e0c44 P e286e4695ccd4ff48b17493f78c4629d: stopping tablet replica
I20260811 12:02:47.083386 14211 raft_consensus.cc:2229] T 012f6ff7436c4098b3dcde5b687e0c44 P e286e4695ccd4ff48b17493f78c4629d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:02:47.083875 14211 raft_consensus.cc:2258] T 012f6ff7436c4098b3dcde5b687e0c44 P e286e4695ccd4ff48b17493f78c4629d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:02:47.085151 14211 ts_tablet_manager.cc:1905] T 012f6ff7436c4098b3dcde5b687e0c44 P e286e4695ccd4ff48b17493f78c4629d: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 12:02:47 dist-test-slave-mjjr krb5kdc[12997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449764, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.221.129@KRBTEST.COM
I20260811 12:02:47.095122 14211 ts_tablet_manager.cc:1918] T 012f6ff7436c4098b3dcde5b687e0c44 P e286e4695ccd4ff48b17493f78c4629d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 12:02:47.095412 14211 log.cc:1196] T 012f6ff7436c4098b3dcde5b687e0c44 P e286e4695ccd4ff48b17493f78c4629d: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449743084027-10102-0/minicluster-data/ts-0/wal/wals/012f6ff7436c4098b3dcde5b687e0c44
I20260811 12:02:47.096259 14221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:47.050835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37728 (local address 127.9.221.129:37553)
0811 12:02:47.074084 (+ 23249us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:47.074094 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:47.074125 (+ 31us) server_negotiation.cc:407] Connection header received
0811 12:02:47.074200 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:47.074208 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:47.074324 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:47.074466 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:47.077153 (+ 2687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:47.078460 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:47.080693 (+ 2233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:47.081513 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:47.081659 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:47.087420 (+ 5761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:47.087434 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:47.087451 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:47.087485 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:47.092476 (+ 4991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:47.093288 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:47.093295 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:47.093298 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:47.093444 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:47.093882 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:47.093888 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:47.093891 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:47.094436 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:47.094921 (+ 485us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:47.095249 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 12:02:47.095565 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23074,"thread_start_us":24348,"threads_started":1}
I20260811 12:02:47.096606 14211 ts_tablet_manager.cc:1939] T 012f6ff7436c4098b3dcde5b687e0c44 P e286e4695ccd4ff48b17493f78c4629d: Deleting consensus metadata
W20260811 12:02:47.097088 13422 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37728
I20260811 12:02:47.097822 13048 catalog_manager.cc:4841] TS e286e4695ccd4ff48b17493f78c4629d (127.9.221.129:37553): tablet 012f6ff7436c4098b3dcde5b687e0c44 (table test-table [id=7d2f42576e3441af80be864edbf26cb8]) successfully deleted
W20260811 12:02:47.097862 10102 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 13203: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:02:47.098191 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 13203
I20260811 12:02:47.140137 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 13432
Aug 11 12:02:47 dist-test-slave-mjjr krb5kdc[12997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449764, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.221.130@KRBTEST.COM
I20260811 12:02:47.166059 14222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:47.141276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49024 (local address 127.9.221.130:38365)
0811 12:02:47.143974 (+ 2698us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:47.143978 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:47.143994 (+ 16us) server_negotiation.cc:407] Connection header received
0811 12:02:47.144048 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:47.144052 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:47.144116 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:47.144206 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:47.147246 (+ 3040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:47.148137 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:47.150397 (+ 2260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:47.150814 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:47.150916 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:47.157342 (+ 6426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:47.157366 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:47.157391 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:47.157452 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:47.162147 (+ 4695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:47.163131 (+ 984us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:47.163142 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:47.163146 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:47.163316 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:47.164075 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:47.164081 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:47.164085 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:47.164508 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:47.165007 (+ 499us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:47.165347 (+ 340us) server_negotiation.cc:299] Negotiation successful
0811 12:02:47.165602 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2570,"spinlock_wait_cycles":14080,"thread_start_us":2543,"threads_started":1}
W20260811 12:02:47.166819 13650 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49024
W20260811 12:02:47.167502 10102 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 13432: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:02:47.167771 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 13432
I20260811 12:02:47.199488 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 13672
Aug 11 12:02:47 dist-test-slave-mjjr krb5kdc[12997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449764, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.221.131@KRBTEST.COM
I20260811 12:02:47.214820 14224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:47.200395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44740 (local address 127.9.221.131:41221)
0811 12:02:47.200904 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:47.200908 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:47.200926 (+ 18us) server_negotiation.cc:407] Connection header received
0811 12:02:47.200982 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:47.200986 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:47.201050 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:47.201139 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:47.202534 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:47.203269 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:47.204614 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:47.205033 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:47.205122 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:47.209467 (+ 4345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:47.209482 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:47.209497 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:47.209528 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:47.212447 (+ 2919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:47.213010 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:47.213017 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:47.213020 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:47.213152 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:47.213465 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:47.213468 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:47.213470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:47.213813 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:47.214040 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:47.214259 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 12:02:47.214421 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":253,"threads_started":1}
W20260811 12:02:47.215524 13901 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44740
W20260811 12:02:47.216178 10102 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 13672: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:02:47.216359 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 13672
I20260811 12:02:47.250798 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 13022
I20260811 12:02:47.252041 13151 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:02:47.370450 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 13022
2026-08-11T12:02:47Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8973 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:02:47 dist-test-slave-mjjr krb5kdc[14230](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:02:47 dist-test-slave-mjjr krb5kdc[14230](info): set up 2 sockets
Aug 11 12:02:47 dist-test-slave-mjjr krb5kdc[14230](info): commencing operation
krb5kdc: starting...
W20260811 12:02:48.703550 10102 mini_kdc.cc:121] Time spent starting KDC: real 1.282s user 0.002s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Aug 11 12:02:48 dist-test-slave-mjjr krb5kdc[14230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449768, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:02:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:02:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.221.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
I20260811 12:02:48.906879 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:35561
--webserver_interface=127.9.221.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:41807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:35561
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:02:49.234683 14252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:02:49.234972 14252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:02:49.235100 14252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:02:49.242334 14252 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:02:49.242449 14252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:02:49.242503 14252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:02:49.242600 14252 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:02:49.242691 14252 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:02:49.252912 14252 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:41807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:35561
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:35561
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.221.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:02:49.254323 14252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:02:49.255743 14252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:02:49.268642 14258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:49.268685 14271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:49.268720 14260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:02:49.269402 14252 server_base.cc:1034] running on GCE node
I20260811 12:02:49.270190 14252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:02:49.271765 14252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:02:49.272990 14252 hybrid_clock.cc:648] HybridClock initialized: now 1786449769272941 us; error 49 us; skew 500 ppm
I20260811 12:02:49.273375 14252 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:02:49 dist-test-slave-mjjr krb5kdc[14230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449769, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:02:49.276821 14252 init.cc:377] Logged in from keytab as kudu/127.9.221.190@KRBTEST.COM (short username kudu)
I20260811 12:02:49.278348 14252 webserver.cc:460] Webserver started at http://127.9.221.190:37333/ using document root <none> and password file <none>
I20260811 12:02:49.279093 14252 fs_manager.cc:359] Metadata directory not provided
I20260811 12:02:49.279232 14252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:02:49.279585 14252 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:02:49.282994 14252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "595aac22cd4e4654b7a7682548a1013d"
format_stamp: "Formatted at 2026-08-11 12:02:49 on dist-test-slave-mjjr"
server_key: "c969ca850a0343955871fb31b62126bd"
server_key_iv: "43e8b329eac4f7bc6393c0deb4e7f2ff"
server_key_version: "encryptionkey@0"
I20260811 12:02:49.283847 14252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "595aac22cd4e4654b7a7682548a1013d"
format_stamp: "Formatted at 2026-08-11 12:02:49 on dist-test-slave-mjjr"
server_key: "c969ca850a0343955871fb31b62126bd"
server_key_iv: "43e8b329eac4f7bc6393c0deb4e7f2ff"
server_key_version: "encryptionkey@0"
I20260811 12:02:49.289621 14252 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 12:02:49.293381 14281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:49.294955 14252 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 12:02:49.295128 14252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "595aac22cd4e4654b7a7682548a1013d"
format_stamp: "Formatted at 2026-08-11 12:02:49 on dist-test-slave-mjjr"
server_key: "c969ca850a0343955871fb31b62126bd"
server_key_iv: "43e8b329eac4f7bc6393c0deb4e7f2ff"
server_key_version: "encryptionkey@0"
I20260811 12:02:49.295360 14252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:02:49.308815 14252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:02:49.313650 14252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:02:49.313906 14252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:02:49.334555 14252 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.190:35561
I20260811 12:02:49.334812 14369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.190:35561 every 8 connection(s)
I20260811 12:02:49.336133 14252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:02:49.340778 14370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:02:49.345944 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 14252
I20260811 12:02:49.346292 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:02:49.347159 10102 external_mini_cluster.cc:1428] Setting key e343e0af202969bf725bd11b9c0b0c97
I20260811 12:02:49.357071 14370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 595aac22cd4e4654b7a7682548a1013d: Bootstrap starting.
I20260811 12:02:49.363441 14370 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 595aac22cd4e4654b7a7682548a1013d: Neither blocks nor log segments found. Creating new log.
I20260811 12:02:49.365535 14370 log.cc:824] T 00000000000000000000000000000000 P 595aac22cd4e4654b7a7682548a1013d: Log is configured to *not* fsync() on all Append() calls
Aug 11 12:02:49 dist-test-slave-mjjr krb5kdc[14230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449768, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:02:49.370769 14370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 595aac22cd4e4654b7a7682548a1013d: No bootstrap required, opened a new log
I20260811 12:02:49.383265 14375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:49.350151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36082 (local address 127.9.221.190:35561)
0811 12:02:49.351853 (+ 1702us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:49.351868 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:49.351930 (+ 62us) server_negotiation.cc:407] Connection header received
0811 12:02:49.355276 (+ 3346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:49.355298 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:49.355766 (+ 468us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:49.356221 (+ 455us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:49.359236 (+ 3015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:49.361264 (+ 2028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:49.364178 (+ 2914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:49.365272 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:49.365466 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:49.372289 (+ 6823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:49.372298 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:49.372311 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:49.372349 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:49.377450 (+ 5101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:49.378789 (+ 1339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:49.378799 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:49.378811 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:49.379187 (+ 376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:49.379734 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:49.379742 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:49.379749 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:49.380668 (+ 919us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:49.380982 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:49.381472 (+ 490us) server_negotiation.cc:299] Negotiation successful
0811 12:02:49.381831 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1389,"thread_start_us":889,"threads_started":1}
I20260811 12:02:49.384680 14370 raft_consensus.cc:348] T 00000000000000000000000000000000 P 595aac22cd4e4654b7a7682548a1013d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "595aac22cd4e4654b7a7682548a1013d" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 35561 } }
I20260811 12:02:49.385360 14370 raft_consensus.cc:374] T 00000000000000000000000000000000 P 595aac22cd4e4654b7a7682548a1013d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:02:49.385545 14370 raft_consensus.cc:729] T 00000000000000000000000000000000 P 595aac22cd4e4654b7a7682548a1013d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 595aac22cd4e4654b7a7682548a1013d, State: Initialized, Role: FOLLOWER
I20260811 12:02:49.386250 14370 consensus_queue.cc:260] T 00000000000000000000000000000000 P 595aac22cd4e4654b7a7682548a1013d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "595aac22cd4e4654b7a7682548a1013d" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 35561 } }
I20260811 12:02:49.386631 14370 raft_consensus.cc:388] T 00000000000000000000000000000000 P 595aac22cd4e4654b7a7682548a1013d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:02:49.386929 14370 raft_consensus.cc:482] T 00000000000000000000000000000000 P 595aac22cd4e4654b7a7682548a1013d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:02:49.387183 14370 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 595aac22cd4e4654b7a7682548a1013d [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:02:49.480290 14370 raft_consensus.cc:504] T 00000000000000000000000000000000 P 595aac22cd4e4654b7a7682548a1013d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "595aac22cd4e4654b7a7682548a1013d" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 35561 } }
I20260811 12:02:49.481130 14370 leader_election.cc:302] T 00000000000000000000000000000000 P 595aac22cd4e4654b7a7682548a1013d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 595aac22cd4e4654b7a7682548a1013d; no voters:
I20260811 12:02:49.482515 14370 leader_election.cc:288] T 00000000000000000000000000000000 P 595aac22cd4e4654b7a7682548a1013d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:02:49.482877 14382 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 595aac22cd4e4654b7a7682548a1013d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:02:49.484956 14382 raft_consensus.cc:686] T 00000000000000000000000000000000 P 595aac22cd4e4654b7a7682548a1013d [term 1 LEADER]: Becoming Leader. State: Replica: 595aac22cd4e4654b7a7682548a1013d, State: Running, Role: LEADER
I20260811 12:02:49.485910 14382 consensus_queue.cc:237] T 00000000000000000000000000000000 P 595aac22cd4e4654b7a7682548a1013d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "595aac22cd4e4654b7a7682548a1013d" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 35561 } }
I20260811 12:02:49.486446 14370 sys_catalog.cc:564] T 00000000000000000000000000000000 P 595aac22cd4e4654b7a7682548a1013d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:02:49.490578 14383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 595aac22cd4e4654b7a7682548a1013d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "595aac22cd4e4654b7a7682548a1013d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "595aac22cd4e4654b7a7682548a1013d" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 35561 } } }
I20260811 12:02:49.491299 14383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 595aac22cd4e4654b7a7682548a1013d [sys.catalog]: This master's current role is: LEADER
I20260811 12:02:49.495179 14384 sys_catalog.cc:455] T 00000000000000000000000000000000 P 595aac22cd4e4654b7a7682548a1013d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 595aac22cd4e4654b7a7682548a1013d. Latest consensus state: current_term: 1 leader_uuid: "595aac22cd4e4654b7a7682548a1013d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "595aac22cd4e4654b7a7682548a1013d" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 35561 } } }
I20260811 12:02:49.496842 14384 sys_catalog.cc:458] T 00000000000000000000000000000000 P 595aac22cd4e4654b7a7682548a1013d [sys.catalog]: This master's current role is: LEADER
I20260811 12:02:49.504586 14394 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:02:49.511994 14394 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:02:49.523195 14394 catalog_manager.cc:1347] Generated new cluster ID: 4fea516aefc243009940027d5dd0b7de
I20260811 12:02:49.523398 14394 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:02:49.551458 14394 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:02:49.554545 14394 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:02:49.572837 14394 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 595aac22cd4e4654b7a7682548a1013d: Generated new TSK 0
I20260811 12:02:49.573849 14394 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.221.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab.
Entry for principal kudu/127.9.221.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab.
W20260811 12:02:49.880359 10102 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.9.221.129: real 0.292s user 0.000s sys 0.013s
I20260811 12:02:49.881352 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.129:0
--local_ip_for_outbound_sockets=127.9.221.129
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:35561
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab
--principal=kudu/127.9.221.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:41807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:02:50.223251 14425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:02:50.223519 14425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:02:50.223613 14425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:02:50.230827 14425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:02:50.230939 14425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:02:50.231109 14425 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.129
I20260811 12:02:50.240978 14425 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:41807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab
--principal=kudu/127.9.221.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:35561
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.129
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:02:50.242347 14425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:02:50.243885 14425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:02:50.257282 14434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:50.257405 14436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:50.257565 14440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:02:50.258551 14425 server_base.cc:1034] running on GCE node
I20260811 12:02:50.259169 14425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:02:50.260140 14425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:02:50.261366 14425 hybrid_clock.cc:648] HybridClock initialized: now 1786449770261332 us; error 51 us; skew 500 ppm
I20260811 12:02:50.261675 14425 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:02:50 dist-test-slave-mjjr krb5kdc[14230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449770, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:02:50.264768 14425 init.cc:377] Logged in from keytab as kudu/127.9.221.129@KRBTEST.COM (short username kudu)
I20260811 12:02:50.266196 14425 webserver.cc:460] Webserver started at http://127.9.221.129:39735/ using document root <none> and password file <none>
I20260811 12:02:50.266819 14425 fs_manager.cc:359] Metadata directory not provided
I20260811 12:02:50.266917 14425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:02:50.267182 14425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:02:50.269989 14425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-0/data/instance:
uuid: "7b0bddd991ef470db57b4d0d9cf620b6"
format_stamp: "Formatted at 2026-08-11 12:02:50 on dist-test-slave-mjjr"
server_key: "362d539fbe405f5bf6c07a53b86826cf"
server_key_iv: "70c891639ef42260abb20490a73e16b5"
server_key_version: "encryptionkey@0"
I20260811 12:02:50.270701 14425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance:
uuid: "7b0bddd991ef470db57b4d0d9cf620b6"
format_stamp: "Formatted at 2026-08-11 12:02:50 on dist-test-slave-mjjr"
server_key: "362d539fbe405f5bf6c07a53b86826cf"
server_key_iv: "70c891639ef42260abb20490a73e16b5"
server_key_version: "encryptionkey@0"
I20260811 12:02:50.275622 14425 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260811 12:02:50.279027 14457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:50.280434 14425 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 12:02:50.280567 14425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-0/wal
uuid: "7b0bddd991ef470db57b4d0d9cf620b6"
format_stamp: "Formatted at 2026-08-11 12:02:50 on dist-test-slave-mjjr"
server_key: "362d539fbe405f5bf6c07a53b86826cf"
server_key_iv: "70c891639ef42260abb20490a73e16b5"
server_key_version: "encryptionkey@0"
I20260811 12:02:50.280720 14425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:02:50.305477 14425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:02:50.309265 14425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:02:50.309500 14425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:02:50.310374 14425 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:02:50.311621 14425 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:02:50.311713 14425 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:50.311794 14425 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:02:50.311838 14425 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:50.345943 14425 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.129:34285
I20260811 12:02:50.345981 14665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.129:34285 every 8 connection(s)
I20260811 12:02:50.347496 14425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
I20260811 12:02:50.355079 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 14425
I20260811 12:02:50.355312 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance
I20260811 12:02:50.356205 10102 external_mini_cluster.cc:1428] Setting key 1c0779b5946a7571dcea507992420ce5
Aug 11 12:02:50 dist-test-slave-mjjr krb5kdc[14230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449770, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:02:50.369896 14669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:50.350148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:54393 (local address 127.9.221.190:35561)
0811 12:02:50.350678 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:50.350685 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:50.351703 (+ 1018us) server_negotiation.cc:407] Connection header received
0811 12:02:50.353520 (+ 1817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:50.353525 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:50.353650 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:50.353778 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:50.355683 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:50.356551 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:50.358695 (+ 2144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:50.359110 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:50.359250 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:50.363308 (+ 4058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:50.363313 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:50.363316 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:50.363348 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:50.366594 (+ 3246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:50.367292 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:50.367300 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:50.367307 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:50.367458 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:50.367914 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:50.367922 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:50.367935 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:50.368323 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:50.368495 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:50.369132 (+ 637us) server_negotiation.cc:299] Negotiation successful
0811 12:02:50.369384 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":280,"threads_started":1}
I20260811 12:02:50.370565 14670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:50.350693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:35561 (local address 127.9.221.129:54393)
0811 12:02:50.351518 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:50.351556 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:50.353243 (+ 1687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:50.354020 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:50.354029 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:50.354576 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:02:50.355426 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:50.355467 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:50.356705 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:50.356710 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:50.358536 (+ 1826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:50.358548 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:50.359338 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:50.359347 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:50.359618 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:50.360369 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:02:50.360390 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:02:50.363094 (+ 2704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:02:50.366733 (+ 3639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:50.366739 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:50.366751 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:50.367174 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:50.367565 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:50.367568 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:50.367571 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:50.367810 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:50.368459 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:02:50.368465 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:02:50.368759 (+ 294us) client_negotiation.cc:769] Sending connection context
0811 12:02:50.368977 (+ 218us) client_negotiation.cc:240] Negotiation successful
0811 12:02:50.369345 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":352,"threads_started":1}
I20260811 12:02:50.372241 14667 heartbeater.cc:347] Connected to a master server at 127.9.221.190:35561
I20260811 12:02:50.372566 14667 heartbeater.cc:464] Registering TS with master...
I20260811 12:02:50.373263 14667 heartbeater.cc:511] Master 127.9.221.190:35561 requested a full tablet report, sending...
I20260811 12:02:50.375753 14309 ts_manager.cc:194] Registered new tserver with Master: 7b0bddd991ef470db57b4d0d9cf620b6 (127.9.221.129:34285)
I20260811 12:02:50.377943 14309 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.129@KRBTEST.COM'} at 127.9.221.129:54393
WARNING: no policy specified for kudu/127.9.221.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab.
Entry for principal kudu/127.9.221.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab.
I20260811 12:02:50.403622 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.130:0
--local_ip_for_outbound_sockets=127.9.221.130
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:35561
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab
--principal=kudu/127.9.221.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:41807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:02:50.736699 14674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:02:50.737016 14674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:02:50.737186 14674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:02:50.744633 14674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:02:50.744760 14674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:02:50.744956 14674 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.130
I20260811 12:02:50.755113 14674 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:41807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab
--principal=kudu/127.9.221.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:35561
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.130
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:02:50.756551 14674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:02:50.757928 14674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:02:50.771072 14682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:50.771534 14687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:50.771605 14683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:02:50.772421 14674 server_base.cc:1034] running on GCE node
I20260811 12:02:50.773017 14674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:02:50.773878 14674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:02:50.775115 14674 hybrid_clock.cc:648] HybridClock initialized: now 1786449770775062 us; error 41 us; skew 500 ppm
I20260811 12:02:50.775470 14674 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:02:50 dist-test-slave-mjjr krb5kdc[14230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449770, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:02:50.778611 14674 init.cc:377] Logged in from keytab as kudu/127.9.221.130@KRBTEST.COM (short username kudu)
I20260811 12:02:50.780187 14674 webserver.cc:460] Webserver started at http://127.9.221.130:33433/ using document root <none> and password file <none>
I20260811 12:02:50.780869 14674 fs_manager.cc:359] Metadata directory not provided
I20260811 12:02:50.781020 14674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:02:50.781339 14674 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:02:50.784341 14674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-1/data/instance:
uuid: "c6076bab21644f148ba3e0d64e01996c"
format_stamp: "Formatted at 2026-08-11 12:02:50 on dist-test-slave-mjjr"
server_key: "6423ab599774b9a7774c69479504b772"
server_key_iv: "2295b9fb3a44a29bfcc20de57f2cbdea"
server_key_version: "encryptionkey@0"
I20260811 12:02:50.785207 14674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance:
uuid: "c6076bab21644f148ba3e0d64e01996c"
format_stamp: "Formatted at 2026-08-11 12:02:50 on dist-test-slave-mjjr"
server_key: "6423ab599774b9a7774c69479504b772"
server_key_iv: "2295b9fb3a44a29bfcc20de57f2cbdea"
server_key_version: "encryptionkey@0"
I20260811 12:02:50.790396 14674 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:02:50.794189 14703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:50.795615 14674 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 12:02:50.795873 14674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-1/wal
uuid: "c6076bab21644f148ba3e0d64e01996c"
format_stamp: "Formatted at 2026-08-11 12:02:50 on dist-test-slave-mjjr"
server_key: "6423ab599774b9a7774c69479504b772"
server_key_iv: "2295b9fb3a44a29bfcc20de57f2cbdea"
server_key_version: "encryptionkey@0"
I20260811 12:02:50.796077 14674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:02:50.809793 14674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:02:50.813935 14674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:02:50.814241 14674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:02:50.815224 14674 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:02:50.816669 14674 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:02:50.816776 14674 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:50.816932 14674 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:02:50.817010 14674 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:50.851341 14674 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.130:44089
I20260811 12:02:50.851392 14899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.130:44089 every 8 connection(s)
I20260811 12:02:50.852762 14674 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
I20260811 12:02:50.853070 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 14674
I20260811 12:02:50.853241 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance
I20260811 12:02:50.853866 10102 external_mini_cluster.cc:1428] Setting key 4e098173bd5e938d5d66436dbf2e9d58
Aug 11 12:02:50 dist-test-slave-mjjr krb5kdc[14230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449770, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.130@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:02:50.876669 14669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:50.855636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:40547 (local address 127.9.221.190:35561)
0811 12:02:50.855852 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:50.855858 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:50.857344 (+ 1486us) server_negotiation.cc:407] Connection header received
0811 12:02:50.860292 (+ 2948us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:50.860297 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:50.860366 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:50.860461 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:50.862598 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:50.863314 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:50.865095 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:50.865906 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:50.866059 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:50.869529 (+ 3470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:50.869538 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:50.869545 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:50.869591 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:50.873308 (+ 3717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:50.874026 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:50.874034 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:50.874040 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:50.874215 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:50.874655 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:50.874662 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:50.874665 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:50.875039 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:50.875221 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:50.876010 (+ 789us) server_negotiation.cc:299] Negotiation successful
0811 12:02:50.876276 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
WARNING: no policy specified for kudu/127.9.221.131@KRBTEST.COM; defaulting to no policy
I20260811 12:02:50.877410 14903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:50.856292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:35561 (local address 127.9.221.130:40547)
0811 12:02:50.857148 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:50.857212 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:50.859971 (+ 2759us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:50.860747 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:50.860762 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:50.861562 (+ 800us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:02:50.862387 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:50.862414 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:50.863485 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:50.863494 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:50.864944 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:50.864958 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:50.866117 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:50.866121 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:50.866312 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:50.867087 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:02:50.867109 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:02:50.869304 (+ 2195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:02:50.873467 (+ 4163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:50.873474 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:50.873486 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:50.873884 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:50.874327 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:50.874330 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:50.874332 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:50.874527 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:50.875203 (+ 676us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:02:50.875209 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:02:50.875543 (+ 334us) client_negotiation.cc:769] Sending connection context
0811 12:02:50.875853 (+ 310us) client_negotiation.cc:240] Negotiation successful
0811 12:02:50.876251 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":303,"threads_started":1}
I20260811 12:02:50.878964 14900 heartbeater.cc:347] Connected to a master server at 127.9.221.190:35561
I20260811 12:02:50.879274 14900 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.131@KRBTEST.COM" created.
I20260811 12:02:50.880205 14900 heartbeater.cc:511] Master 127.9.221.190:35561 requested a full tablet report, sending...
I20260811 12:02:50.881908 14309 ts_manager.cc:194] Registered new tserver with Master: c6076bab21644f148ba3e0d64e01996c (127.9.221.130:44089)
I20260811 12:02:50.882759 14309 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.130@KRBTEST.COM'} at 127.9.221.130:40547
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab.
Entry for principal kudu/127.9.221.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab.
I20260811 12:02:50.905045 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.131:0
--local_ip_for_outbound_sockets=127.9.221.131
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:35561
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab
--principal=kudu/127.9.221.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:41807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:02:51.230288 14906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:02:51.230607 14906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:02:51.230791 14906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:02:51.238514 14906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:02:51.238669 14906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:02:51.238847 14906 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.131
I20260811 12:02:51.248270 14906 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:41807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab
--principal=kudu/127.9.221.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:35561
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.131
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:02:51.249550 14906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:02:51.250847 14906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:02:51.263862 14915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:51.263828 14925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:51.263893 14919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:02:51.265067 14906 server_base.cc:1034] running on GCE node
I20260811 12:02:51.265653 14906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:02:51.266541 14906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:02:51.267750 14906 hybrid_clock.cc:648] HybridClock initialized: now 1786449771267677 us; error 48 us; skew 500 ppm
I20260811 12:02:51.268083 14906 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:02:51 dist-test-slave-mjjr krb5kdc[14230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449771, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:02:51.271276 14906 init.cc:377] Logged in from keytab as kudu/127.9.221.131@KRBTEST.COM (short username kudu)
I20260811 12:02:51.272876 14906 webserver.cc:460] Webserver started at http://127.9.221.131:37747/ using document root <none> and password file <none>
I20260811 12:02:51.273598 14906 fs_manager.cc:359] Metadata directory not provided
I20260811 12:02:51.273754 14906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:02:51.274080 14906 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:02:51.277153 14906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-2/data/instance:
uuid: "394a4703006b4225ab46cc508b128de2"
format_stamp: "Formatted at 2026-08-11 12:02:51 on dist-test-slave-mjjr"
server_key: "1ca6fa3b27ecc2b1af2d8a37d45c56be"
server_key_iv: "9fabb126564ed5dc84bd9f23b510a5eb"
server_key_version: "encryptionkey@0"
I20260811 12:02:51.278000 14906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance:
uuid: "394a4703006b4225ab46cc508b128de2"
format_stamp: "Formatted at 2026-08-11 12:02:51 on dist-test-slave-mjjr"
server_key: "1ca6fa3b27ecc2b1af2d8a37d45c56be"
server_key_iv: "9fabb126564ed5dc84bd9f23b510a5eb"
server_key_version: "encryptionkey@0"
I20260811 12:02:51.283392 14906 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 12:02:51.287341 14937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:51.288791 14906 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:02:51.288988 14906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-2/wal
uuid: "394a4703006b4225ab46cc508b128de2"
format_stamp: "Formatted at 2026-08-11 12:02:51 on dist-test-slave-mjjr"
server_key: "1ca6fa3b27ecc2b1af2d8a37d45c56be"
server_key_iv: "9fabb126564ed5dc84bd9f23b510a5eb"
server_key_version: "encryptionkey@0"
I20260811 12:02:51.289211 14906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:02:51.339532 14906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:02:51.345098 14906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:02:51.345499 14906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:02:51.346675 14906 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:02:51.348172 14906 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:02:51.348289 14906 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:51.348459 14906 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:02:51.348537 14906 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:51.381213 14667 heartbeater.cc:503] Master 127.9.221.190:35561 was elected leader, sending a full tablet report...
I20260811 12:02:51.384902 14906 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.131:39487
I20260811 12:02:51.384955 15131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.131:39487 every 8 connection(s)
I20260811 12:02:51.386219 14906 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
I20260811 12:02:51.394315 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 14906
I20260811 12:02:51.394495 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance
I20260811 12:02:51.395040 10102 external_mini_cluster.cc:1428] Setting key 368cd0110dc6e89b8507a01dfe767c94
Aug 11 12:02:51 dist-test-slave-mjjr krb5kdc[14230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449771, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.131@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:02:51.407029 15138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:51.388916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:34791 (local address 127.9.221.190:35561)
0811 12:02:51.389361 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:51.389365 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:51.390201 (+ 836us) server_negotiation.cc:407] Connection header received
0811 12:02:51.392093 (+ 1892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:51.392099 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:51.392195 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:51.392313 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:51.393798 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:51.394986 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:51.396792 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:51.397376 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:51.397531 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:51.401442 (+ 3911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:51.401448 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:51.401463 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:51.401496 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:51.403970 (+ 2474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:51.404667 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:51.404672 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:51.404676 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:51.404803 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:51.405258 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:51.405262 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:51.405266 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:51.405542 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:51.405736 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:51.406369 (+ 633us) server_negotiation.cc:299] Negotiation successful
0811 12:02:51.406644 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":240,"threads_started":1}
I20260811 12:02:51.408000 15139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:51.389345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:35561 (local address 127.9.221.131:34791)
0811 12:02:51.390040 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:51.390081 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:51.391821 (+ 1740us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:51.392544 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:51.392551 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:51.393065 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:02:51.393624 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:51.393641 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:51.395184 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:51.395189 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:51.396646 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:51.396659 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:51.397658 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:51.397666 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:51.397969 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:51.398753 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:02:51.398775 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:02:51.401224 (+ 2449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:02:51.404097 (+ 2873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:51.404105 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:51.404119 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:51.404536 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:51.404925 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:51.404930 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:51.404933 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:51.405131 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:51.405705 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:02:51.405712 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:02:51.406025 (+ 313us) client_negotiation.cc:769] Sending connection context
0811 12:02:51.406272 (+ 247us) client_negotiation.cc:240] Negotiation successful
0811 12:02:51.406800 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":283,"threads_started":1}
I20260811 12:02:51.409243 15135 heartbeater.cc:347] Connected to a master server at 127.9.221.190:35561
I20260811 12:02:51.409565 15135 heartbeater.cc:464] Registering TS with master...
I20260811 12:02:51.410315 15135 heartbeater.cc:511] Master 127.9.221.190:35561 requested a full tablet report, sending...
I20260811 12:02:51.411626 14309 ts_manager.cc:194] Registered new tserver with Master: 394a4703006b4225ab46cc508b128de2 (127.9.221.131:39487)
I20260811 12:02:51.412492 14309 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.131@KRBTEST.COM'} at 127.9.221.131:34791
I20260811 12:02:51.422487 10102 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 12:02:51.456950 15138 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:02:51.454518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36098 (local address 127.9.221.190:35561)
0811 12:02:51.454785 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:51.454792 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:51.455030 (+ 238us) server_negotiation.cc:407] Connection header received
0811 12:02:51.455180 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:51.455184 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:51.455263 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:51.455348 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:51.456667 (+ 1319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36098: BlockingRecv error: recv got EOF from 127.0.0.1:36098 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 12:02:51.460454 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 14425
W20260811 12:02:51.463065 10102 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 14425: Not authorized: Client connection negotiation failed: client connection to 127.9.221.129:34285: server requires authentication, but client does not have Kerberos credentials available
I20260811 12:02:51.463239 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 14425
I20260811 12:02:51.487013 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 14674
W20260811 12:02:51.489833 10102 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 14674: Not authorized: Client connection negotiation failed: client connection to 127.9.221.130:44089: server requires authentication, but client does not have Kerberos credentials available
I20260811 12:02:51.489948 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 14674
I20260811 12:02:51.516615 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 14906
W20260811 12:02:51.519518 10102 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 14906: Not authorized: Client connection negotiation failed: client connection to 127.9.221.131:39487: server requires authentication, but client does not have Kerberos credentials available
I20260811 12:02:51.519718 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 14906
I20260811 12:02:51.542866 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 14252
I20260811 12:02:51.544176 14362 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:02:51.650116 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 14252
2026-08-11T12:02:51Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (4280 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449743084027-10102-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:02:51 dist-test-slave-mjjr krb5kdc[15168](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:02:51 dist-test-slave-mjjr krb5kdc[15168](info): set up 2 sockets
Aug 11 12:02:51 dist-test-slave-mjjr krb5kdc[15168](info): commencing operation
krb5kdc: starting...
W20260811 12:02:52.969472 10102 mini_kdc.cc:121] Time spent starting KDC: real 1.268s user 0.004s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Aug 11 12:02:53 dist-test-slave-mjjr krb5kdc[15168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:02:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:02:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.221.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
I20260811 12:02:53.173174 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:32979
--webserver_interface=127.9.221.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:36379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:32979
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:02:53.499697 15199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:02:53.500000 15199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:02:53.500174 15199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:02:53.507797 15199 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:02:53.507916 15199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:02:53.508004 15199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:02:53.508109 15199 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:02:53.508183 15199 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:02:53.517565 15199 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:36379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:32979
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:32979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.221.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:02:53.518783 15199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:02:53.520100 15199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:02:53.532246 15208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:53.532217 15215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:53.532301 15211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:02:53.533393 15199 server_base.cc:1034] running on GCE node
I20260811 12:02:53.534086 15199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:02:53.535375 15199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:02:53.536576 15199 hybrid_clock.cc:648] HybridClock initialized: now 1786449773536515 us; error 52 us; skew 500 ppm
I20260811 12:02:53.536933 15199 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:02:53 dist-test-slave-mjjr krb5kdc[15168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449773, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:02:53.543679 15199 init.cc:377] Logged in from keytab as kudu/127.9.221.190@KRBTEST.COM (short username kudu)
I20260811 12:02:53.545162 15199 webserver.cc:460] Webserver started at http://127.9.221.190:32973/ using document root <none> and password file <none>
I20260811 12:02:53.545825 15199 fs_manager.cc:359] Metadata directory not provided
I20260811 12:02:53.545958 15199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:02:53.546278 15199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:02:53.549273 15199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "f450647c706f40dbb33573fd743bc2c4"
format_stamp: "Formatted at 2026-08-11 12:02:53 on dist-test-slave-mjjr"
server_key: "5b0ba855a7ffdba5b8a34acdbaa28e85"
server_key_iv: "87f4f72aff0a92b634d28f6dcd571c0d"
server_key_version: "encryptionkey@0"
I20260811 12:02:53.550058 15199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "f450647c706f40dbb33573fd743bc2c4"
format_stamp: "Formatted at 2026-08-11 12:02:53 on dist-test-slave-mjjr"
server_key: "5b0ba855a7ffdba5b8a34acdbaa28e85"
server_key_iv: "87f4f72aff0a92b634d28f6dcd571c0d"
server_key_version: "encryptionkey@0"
I20260811 12:02:53.555043 15199 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:02:53.558303 15227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:53.559723 15199 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 12:02:53.559883 15199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "f450647c706f40dbb33573fd743bc2c4"
format_stamp: "Formatted at 2026-08-11 12:02:53 on dist-test-slave-mjjr"
server_key: "5b0ba855a7ffdba5b8a34acdbaa28e85"
server_key_iv: "87f4f72aff0a92b634d28f6dcd571c0d"
server_key_version: "encryptionkey@0"
I20260811 12:02:53.560053 15199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:02:53.591082 15199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:02:53.594996 15199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:02:53.595293 15199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:02:53.614246 15199 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.190:32979
I20260811 12:02:53.614281 15316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.190:32979 every 8 connection(s)
I20260811 12:02:53.615633 15199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:02:53.619153 15317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:02:53.620790 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 15199
I20260811 12:02:53.621066 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:02:53.621912 10102 external_mini_cluster.cc:1428] Setting key 7121827f8dd5f18f928960e79088a4af
I20260811 12:02:53.633543 15317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f450647c706f40dbb33573fd743bc2c4: Bootstrap starting.
Aug 11 12:02:53 dist-test-slave-mjjr krb5kdc[15168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:02:53.637290 15317 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f450647c706f40dbb33573fd743bc2c4: Neither blocks nor log segments found. Creating new log.
I20260811 12:02:53.638311 15317 log.cc:824] T 00000000000000000000000000000000 P f450647c706f40dbb33573fd743bc2c4: Log is configured to *not* fsync() on all Append() calls
I20260811 12:02:53.641199 15317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f450647c706f40dbb33573fd743bc2c4: No bootstrap required, opened a new log
I20260811 12:02:53.644908 15324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:53.623843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41852 (local address 127.9.221.190:32979)
0811 12:02:53.624675 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:53.624688 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:53.624730 (+ 42us) server_negotiation.cc:407] Connection header received
0811 12:02:53.626265 (+ 1535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:53.626284 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:53.626680 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:53.626978 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:53.628819 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:53.630123 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:53.631925 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:53.632572 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:53.632696 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:53.637477 (+ 4781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:53.637487 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:53.637501 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:53.637536 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:53.640823 (+ 3287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:53.641566 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:53.641572 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:53.641577 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:53.641742 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:53.642159 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:53.642163 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:53.642166 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:53.642698 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:53.643021 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:53.643531 (+ 510us) server_negotiation.cc:299] Negotiation successful
0811 12:02:53.643962 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":382,"threads_started":1}
I20260811 12:02:53.648645 10102 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:02:53.649251 15317 raft_consensus.cc:348] T 00000000000000000000000000000000 P f450647c706f40dbb33573fd743bc2c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f450647c706f40dbb33573fd743bc2c4" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 32979 } }
I20260811 12:02:53.649622 15317 raft_consensus.cc:374] T 00000000000000000000000000000000 P f450647c706f40dbb33573fd743bc2c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:02:53.649761 15317 raft_consensus.cc:729] T 00000000000000000000000000000000 P f450647c706f40dbb33573fd743bc2c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f450647c706f40dbb33573fd743bc2c4, State: Initialized, Role: FOLLOWER
I20260811 12:02:53.650282 15317 consensus_queue.cc:260] T 00000000000000000000000000000000 P f450647c706f40dbb33573fd743bc2c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f450647c706f40dbb33573fd743bc2c4" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 32979 } }
I20260811 12:02:53.650522 15317 raft_consensus.cc:388] T 00000000000000000000000000000000 P f450647c706f40dbb33573fd743bc2c4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:02:53.650671 15317 raft_consensus.cc:482] T 00000000000000000000000000000000 P f450647c706f40dbb33573fd743bc2c4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:02:53.650831 15317 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f450647c706f40dbb33573fd743bc2c4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:02:53.652977 15317 raft_consensus.cc:504] T 00000000000000000000000000000000 P f450647c706f40dbb33573fd743bc2c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f450647c706f40dbb33573fd743bc2c4" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 32979 } }
I20260811 12:02:53.653461 15317 leader_election.cc:302] T 00000000000000000000000000000000 P f450647c706f40dbb33573fd743bc2c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f450647c706f40dbb33573fd743bc2c4; no voters:
I20260811 12:02:53.654104 15317 leader_election.cc:288] T 00000000000000000000000000000000 P f450647c706f40dbb33573fd743bc2c4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:02:53.654206 15331 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f450647c706f40dbb33573fd743bc2c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:02:53.655918 15331 raft_consensus.cc:686] T 00000000000000000000000000000000 P f450647c706f40dbb33573fd743bc2c4 [term 1 LEADER]: Becoming Leader. State: Replica: f450647c706f40dbb33573fd743bc2c4, State: Running, Role: LEADER
I20260811 12:02:53.656589 15317 sys_catalog.cc:564] T 00000000000000000000000000000000 P f450647c706f40dbb33573fd743bc2c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:02:53.656529 15331 consensus_queue.cc:237] T 00000000000000000000000000000000 P f450647c706f40dbb33573fd743bc2c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f450647c706f40dbb33573fd743bc2c4" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 32979 } }
I20260811 12:02:53.661906 15332 sys_catalog.cc:455] T 00000000000000000000000000000000 P f450647c706f40dbb33573fd743bc2c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f450647c706f40dbb33573fd743bc2c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f450647c706f40dbb33573fd743bc2c4" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 32979 } } }
I20260811 12:02:53.661875 15334 sys_catalog.cc:455] T 00000000000000000000000000000000 P f450647c706f40dbb33573fd743bc2c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f450647c706f40dbb33573fd743bc2c4. Latest consensus state: current_term: 1 leader_uuid: "f450647c706f40dbb33573fd743bc2c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f450647c706f40dbb33573fd743bc2c4" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 32979 } } }
I20260811 12:02:53.662318 15334 sys_catalog.cc:458] T 00000000000000000000000000000000 P f450647c706f40dbb33573fd743bc2c4 [sys.catalog]: This master's current role is: LEADER
I20260811 12:02:53.662318 15332 sys_catalog.cc:458] T 00000000000000000000000000000000 P f450647c706f40dbb33573fd743bc2c4 [sys.catalog]: This master's current role is: LEADER
I20260811 12:02:53.670030 15356 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:02:53.680285 15356 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:02:53.694177 15373 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 12:02:53.695793 15356 catalog_manager.cc:1347] Generated new cluster ID: 89b233b3129b4986aa41c7bf1d3cadd4
I20260811 12:02:53.696029 15356 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:02:53.714679 15324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:53.705432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41856 (local address 127.9.221.190:32979)
0811 12:02:53.705638 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:53.705647 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:53.706162 (+ 515us) server_negotiation.cc:407] Connection header received
0811 12:02:53.706353 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:53.706358 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:53.706432 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:53.706542 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:53.707906 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:53.709001 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:53.710877 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:53.711462 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:53.711588 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:53.712864 (+ 1276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:53.712871 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:53.712875 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:02:53.713825 (+ 950us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:53.713898 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:02:53.713947 (+ 49us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:53.714052 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:53.714228 (+ 176us) server_negotiation.cc:299] Negotiation successful
0811 12:02:53.714325 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 12:02:53.717958 15356 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:02:53.719295 15356 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:02:53.727957 15356 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f450647c706f40dbb33573fd743bc2c4: Generated new TSK 0
I20260811 12:02:53.728739 15356 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:02:53.739773 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 15199
I20260811 12:02:53.740990 15310 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:02:53.831881 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 15199
2026-08-11T12:02:53Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2174 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:02:53 dist-test-slave-mjjr krb5kdc[15395](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:02:53 dist-test-slave-mjjr krb5kdc[15395](info): set up 2 sockets
Aug 11 12:02:53 dist-test-slave-mjjr krb5kdc[15395](info): commencing operation
krb5kdc: starting...
W20260811 12:02:55.583021 10102 mini_kdc.cc:121] Time spent starting KDC: real 1.706s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Aug 11 12:02:55 dist-test-slave-mjjr krb5kdc[15395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449775, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:02:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:02:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.221.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
I20260811 12:02:55.898168 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:33845
--webserver_interface=127.9.221.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:34627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:33845
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:02:56.472246 15418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:02:56.472707 15418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:02:56.472981 15418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:02:56.485061 15418 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:02:56.485283 15418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:02:56.485441 15418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:02:56.485585 15418 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:02:56.485719 15418 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:02:56.503592 15418 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:34627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:33845
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:33845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.221.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:02:56.506161 15418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:02:56.508834 15418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:02:56.529036 15432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:56.529156 15437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:56.529227 15431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:02:56.530854 15418 server_base.cc:1034] running on GCE node
I20260811 12:02:56.532310 15418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:02:56.534632 15418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:02:56.535898 15418 hybrid_clock.cc:648] HybridClock initialized: now 1786449776535816 us; error 70 us; skew 500 ppm
I20260811 12:02:56.536475 15418 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:02:56 dist-test-slave-mjjr krb5kdc[15395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449776, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:02:56.541571 15418 init.cc:377] Logged in from keytab as kudu/127.9.221.190@KRBTEST.COM (short username kudu)
I20260811 12:02:56.544106 15418 webserver.cc:460] Webserver started at http://127.9.221.190:34535/ using document root <none> and password file <none>
I20260811 12:02:56.545202 15418 fs_manager.cc:359] Metadata directory not provided
I20260811 12:02:56.545414 15418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:02:56.545918 15418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:02:56.550848 15418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "05a8c3f8e6f74b2a9217d3f7b8c1c33f"
format_stamp: "Formatted at 2026-08-11 12:02:56 on dist-test-slave-mjjr"
server_key: "ccca97df3b22db4a85879f03012d689e"
server_key_iv: "5c908da966eaf2317c885d15bd823106"
server_key_version: "encryptionkey@0"
I20260811 12:02:56.552173 15418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "05a8c3f8e6f74b2a9217d3f7b8c1c33f"
format_stamp: "Formatted at 2026-08-11 12:02:56 on dist-test-slave-mjjr"
server_key: "ccca97df3b22db4a85879f03012d689e"
server_key_iv: "5c908da966eaf2317c885d15bd823106"
server_key_version: "encryptionkey@0"
I20260811 12:02:56.561162 15418 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.001s
I20260811 12:02:56.567252 15451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:56.569572 15418 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260811 12:02:56.569850 15418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "05a8c3f8e6f74b2a9217d3f7b8c1c33f"
format_stamp: "Formatted at 2026-08-11 12:02:56 on dist-test-slave-mjjr"
server_key: "ccca97df3b22db4a85879f03012d689e"
server_key_iv: "5c908da966eaf2317c885d15bd823106"
server_key_version: "encryptionkey@0"
I20260811 12:02:56.570149 15418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:02:56.604213 15418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:02:56.611078 15418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:02:56.611541 15418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:02:56.646418 15418 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.190:33845
I20260811 12:02:56.646426 15527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.190:33845 every 8 connection(s)
I20260811 12:02:56.649204 15418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:02:56.653244 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 15418
I20260811 12:02:56.653730 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:02:56.655058 10102 external_mini_cluster.cc:1428] Setting key e6e0bdf51108f160afadb5292b0742b4
I20260811 12:02:56.656281 15529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:02:56.684033 15529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05a8c3f8e6f74b2a9217d3f7b8c1c33f: Bootstrap starting.
Aug 11 12:02:56 dist-test-slave-mjjr krb5kdc[15395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449775, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:02:56.691147 15529 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 05a8c3f8e6f74b2a9217d3f7b8c1c33f: Neither blocks nor log segments found. Creating new log.
I20260811 12:02:56.693290 15529 log.cc:824] T 00000000000000000000000000000000 P 05a8c3f8e6f74b2a9217d3f7b8c1c33f: Log is configured to *not* fsync() on all Append() calls
I20260811 12:02:56.699826 15529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05a8c3f8e6f74b2a9217d3f7b8c1c33f: No bootstrap required, opened a new log
I20260811 12:02:56.703796 15536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:56.659375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55370 (local address 127.9.221.190:33845)
0811 12:02:56.660966 (+ 1591us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:56.661008 (+ 42us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:56.661114 (+ 106us) server_negotiation.cc:407] Connection header received
0811 12:02:56.665123 (+ 4009us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:56.665202 (+ 79us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:56.666277 (+ 1075us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:56.667148 (+ 871us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:56.669515 (+ 2367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:56.673266 (+ 3751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:56.676949 (+ 3683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:56.679332 (+ 2383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:56.679781 (+ 449us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:56.689494 (+ 9713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:56.689521 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:56.689545 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:56.689616 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:56.694876 (+ 5260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:56.696340 (+ 1464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:56.696362 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:56.696372 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:56.696821 (+ 449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:56.697417 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:56.697425 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:56.697429 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:56.698824 (+ 1395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:56.699457 (+ 633us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:56.700113 (+ 656us) server_negotiation.cc:299] Negotiation successful
0811 12:02:56.700962 (+ 849us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1101,"thread_start_us":701,"threads_started":1}
I20260811 12:02:56.716166 15529 raft_consensus.cc:348] T 00000000000000000000000000000000 P 05a8c3f8e6f74b2a9217d3f7b8c1c33f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05a8c3f8e6f74b2a9217d3f7b8c1c33f" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 33845 } }
I20260811 12:02:56.716706 15529 raft_consensus.cc:374] T 00000000000000000000000000000000 P 05a8c3f8e6f74b2a9217d3f7b8c1c33f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:02:56.716825 15529 raft_consensus.cc:729] T 00000000000000000000000000000000 P 05a8c3f8e6f74b2a9217d3f7b8c1c33f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a8c3f8e6f74b2a9217d3f7b8c1c33f, State: Initialized, Role: FOLLOWER
I20260811 12:02:56.717669 15529 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05a8c3f8e6f74b2a9217d3f7b8c1c33f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05a8c3f8e6f74b2a9217d3f7b8c1c33f" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 33845 } }
I20260811 12:02:56.718006 15529 raft_consensus.cc:388] T 00000000000000000000000000000000 P 05a8c3f8e6f74b2a9217d3f7b8c1c33f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:02:56.718165 15529 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05a8c3f8e6f74b2a9217d3f7b8c1c33f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:02:56.718408 15529 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05a8c3f8e6f74b2a9217d3f7b8c1c33f [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:02:56.722815 15529 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05a8c3f8e6f74b2a9217d3f7b8c1c33f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05a8c3f8e6f74b2a9217d3f7b8c1c33f" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 33845 } }
I20260811 12:02:56.723755 15529 leader_election.cc:302] T 00000000000000000000000000000000 P 05a8c3f8e6f74b2a9217d3f7b8c1c33f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 05a8c3f8e6f74b2a9217d3f7b8c1c33f; no voters:
I20260811 12:02:56.724637 15529 leader_election.cc:288] T 00000000000000000000000000000000 P 05a8c3f8e6f74b2a9217d3f7b8c1c33f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:02:56.724843 15538 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 05a8c3f8e6f74b2a9217d3f7b8c1c33f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:02:56.727154 15538 raft_consensus.cc:686] T 00000000000000000000000000000000 P 05a8c3f8e6f74b2a9217d3f7b8c1c33f [term 1 LEADER]: Becoming Leader. State: Replica: 05a8c3f8e6f74b2a9217d3f7b8c1c33f, State: Running, Role: LEADER
I20260811 12:02:56.728044 15538 consensus_queue.cc:237] T 00000000000000000000000000000000 P 05a8c3f8e6f74b2a9217d3f7b8c1c33f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05a8c3f8e6f74b2a9217d3f7b8c1c33f" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 33845 } }
I20260811 12:02:56.728633 15529 sys_catalog.cc:564] T 00000000000000000000000000000000 P 05a8c3f8e6f74b2a9217d3f7b8c1c33f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:02:56.733570 15541 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05a8c3f8e6f74b2a9217d3f7b8c1c33f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05a8c3f8e6f74b2a9217d3f7b8c1c33f. Latest consensus state: current_term: 1 leader_uuid: "05a8c3f8e6f74b2a9217d3f7b8c1c33f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05a8c3f8e6f74b2a9217d3f7b8c1c33f" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 33845 } } }
I20260811 12:02:56.734751 15541 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05a8c3f8e6f74b2a9217d3f7b8c1c33f [sys.catalog]: This master's current role is: LEADER
I20260811 12:02:56.735149 15539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05a8c3f8e6f74b2a9217d3f7b8c1c33f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "05a8c3f8e6f74b2a9217d3f7b8c1c33f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05a8c3f8e6f74b2a9217d3f7b8c1c33f" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 33845 } } }
I20260811 12:02:56.737143 15539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05a8c3f8e6f74b2a9217d3f7b8c1c33f [sys.catalog]: This master's current role is: LEADER
I20260811 12:02:56.738152 15552 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:02:56.749552 15552 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:02:56.767510 15552 catalog_manager.cc:1347] Generated new cluster ID: 1708f0a8df2744b981b9f2b6ac01938b
I20260811 12:02:56.767745 15552 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:02:56.785300 15552 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:02:56.787556 15552 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:02:56.806380 15552 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 05a8c3f8e6f74b2a9217d3f7b8c1c33f: Generated new TSK 0
I20260811 12:02:56.808079 15552 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.221.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab.
Entry for principal kudu/127.9.221.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab.
I20260811 12:02:56.907810 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.129:0
--local_ip_for_outbound_sockets=127.9.221.129
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:33845
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab
--principal=kudu/127.9.221.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:34627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:02:57.539008 15569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:02:57.539574 15569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:02:57.539888 15569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:02:57.552300 15569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:02:57.552469 15569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:02:57.552788 15569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.129
I20260811 12:02:57.569384 15569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:34627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab
--principal=kudu/127.9.221.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:33845
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.129
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:02:57.571921 15569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:02:57.574229 15569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:02:57.602885 15587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:57.603012 15590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:57.603807 15583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:02:57.604964 15569 server_base.cc:1034] running on GCE node
I20260811 12:02:57.605880 15569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:02:57.607375 15569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:02:57.608716 15569 hybrid_clock.cc:648] HybridClock initialized: now 1786449777608644 us; error 87 us; skew 500 ppm
I20260811 12:02:57.609344 15569 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:02:57 dist-test-slave-mjjr krb5kdc[15395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449777, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:02:57.614711 15569 init.cc:377] Logged in from keytab as kudu/127.9.221.129@KRBTEST.COM (short username kudu)
I20260811 12:02:57.617133 15569 webserver.cc:460] Webserver started at http://127.9.221.129:43929/ using document root <none> and password file <none>
I20260811 12:02:57.618256 15569 fs_manager.cc:359] Metadata directory not provided
I20260811 12:02:57.618474 15569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:02:57.618949 15569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:02:57.623977 15569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-0/data/instance:
uuid: "7eff7f93219248c597c7ce985700d680"
format_stamp: "Formatted at 2026-08-11 12:02:57 on dist-test-slave-mjjr"
server_key: "f8707b39864392e03961ad58c0812f17"
server_key_iv: "b789893bc0e052cd20a9998845f8357b"
server_key_version: "encryptionkey@0"
I20260811 12:02:57.625336 15569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance:
uuid: "7eff7f93219248c597c7ce985700d680"
format_stamp: "Formatted at 2026-08-11 12:02:57 on dist-test-slave-mjjr"
server_key: "f8707b39864392e03961ad58c0812f17"
server_key_iv: "b789893bc0e052cd20a9998845f8357b"
server_key_version: "encryptionkey@0"
I20260811 12:02:57.633960 15569 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.003s
I20260811 12:02:57.639802 15598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:57.641916 15569 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.002s
I20260811 12:02:57.642150 15569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-0/wal
uuid: "7eff7f93219248c597c7ce985700d680"
format_stamp: "Formatted at 2026-08-11 12:02:57 on dist-test-slave-mjjr"
server_key: "f8707b39864392e03961ad58c0812f17"
server_key_iv: "b789893bc0e052cd20a9998845f8357b"
server_key_version: "encryptionkey@0"
I20260811 12:02:57.642428 15569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:02:57.674671 15569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:02:57.681296 15569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:02:57.681746 15569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:02:57.683307 15569 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:02:57.685469 15569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:02:57.685586 15569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:57.685853 15569 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:02:57.685983 15569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:57.740964 15569 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.129:32877
I20260811 12:02:57.741087 15773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.129:32877 every 8 connection(s)
I20260811 12:02:57.743062 15569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
I20260811 12:02:57.745787 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 15569
I20260811 12:02:57.746214 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance
I20260811 12:02:57.747823 10102 external_mini_cluster.cc:1428] Setting key d25a5113ac69b8ca134b8772eaab053d
Aug 11 12:02:57 dist-test-slave-mjjr krb5kdc[15395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449777, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:02:57.781972 15777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:57.748802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:39553 (local address 127.9.221.190:33845)
0811 12:02:57.749908 (+ 1106us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:57.749913 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:57.751738 (+ 1825us) server_negotiation.cc:407] Connection header received
0811 12:02:57.755036 (+ 3298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:57.755046 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:57.755191 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:57.755436 (+ 245us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:57.757939 (+ 2503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:57.760210 (+ 2271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:57.762905 (+ 2695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:57.763567 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:57.763839 (+ 272us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:57.771237 (+ 7398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:57.771245 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:57.771255 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:57.771308 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:57.775956 (+ 4648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:57.777511 (+ 1555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:57.777520 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:57.777530 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:57.777741 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:57.778548 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:57.778555 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:57.778559 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:57.779039 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:57.779336 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:57.780649 (+ 1313us) server_negotiation.cc:299] Negotiation successful
0811 12:02:57.781116 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":884,"thread_start_us":657,"threads_started":1}
WARNING: no policy specified for kudu/127.9.221.130@KRBTEST.COM; defaulting to no policy
I20260811 12:02:57.783717 15778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:57.749878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:33845 (local address 127.9.221.129:39553)
0811 12:02:57.751342 (+ 1464us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:57.751447 (+ 105us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:57.754591 (+ 3144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:57.755787 (+ 1196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:57.755805 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:57.756634 (+ 829us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:02:57.757654 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:57.757690 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:57.760452 (+ 2762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:57.760464 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:57.762609 (+ 2145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:57.762667 (+ 58us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:57.764048 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:57.764067 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:57.764743 (+ 676us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:57.765986 (+ 1243us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:02:57.766055 (+ 69us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:02:57.770790 (+ 4735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:02:57.776217 (+ 5427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:57.776258 (+ 41us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:57.776298 (+ 40us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:57.777249 (+ 951us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:57.778006 (+ 757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:57.778014 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:57.778028 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:57.778333 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:57.779295 (+ 962us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:02:57.779316 (+ 21us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:02:57.779975 (+ 659us) client_negotiation.cc:769] Sending connection context
0811 12:02:57.780419 (+ 444us) client_negotiation.cc:240] Negotiation successful
0811 12:02:57.781491 (+ 1072us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":966,"thread_start_us":585,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.130@KRBTEST.COM" created.
I20260811 12:02:57.786373 15774 heartbeater.cc:347] Connected to a master server at 127.9.221.190:33845
I20260811 12:02:57.786978 15774 heartbeater.cc:464] Registering TS with master...
I20260811 12:02:57.788602 15774 heartbeater.cc:511] Master 127.9.221.190:33845 requested a full tablet report, sending...
I20260811 12:02:57.791950 15476 ts_manager.cc:194] Registered new tserver with Master: 7eff7f93219248c597c7ce985700d680 (127.9.221.129:32877)
I20260811 12:02:57.794292 15476 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.129@KRBTEST.COM'} at 127.9.221.129:39553
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab.
Entry for principal kudu/127.9.221.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab.
I20260811 12:02:57.819898 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.130:0
--local_ip_for_outbound_sockets=127.9.221.130
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:33845
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab
--principal=kudu/127.9.221.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:34627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:02:58.246572 15785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:02:58.246863 15785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:02:58.247033 15785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:02:58.254305 15785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:02:58.254421 15785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:02:58.254572 15785 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.130
I20260811 12:02:58.264307 15785 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:34627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab
--principal=kudu/127.9.221.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:33845
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.130
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:02:58.265710 15785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:02:58.267051 15785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:02:58.280784 15801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:58.280856 15798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:58.281100 15796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:02:58.281984 15785 server_base.cc:1034] running on GCE node
I20260811 12:02:58.282629 15785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:02:58.283591 15785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:02:58.284840 15785 hybrid_clock.cc:648] HybridClock initialized: now 1786449778284794 us; error 40 us; skew 500 ppm
I20260811 12:02:58.285157 15785 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:02:58 dist-test-slave-mjjr krb5kdc[15395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449778, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:02:58.288496 15785 init.cc:377] Logged in from keytab as kudu/127.9.221.130@KRBTEST.COM (short username kudu)
I20260811 12:02:58.289911 15785 webserver.cc:460] Webserver started at http://127.9.221.130:38243/ using document root <none> and password file <none>
I20260811 12:02:58.290599 15785 fs_manager.cc:359] Metadata directory not provided
I20260811 12:02:58.290748 15785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:02:58.291052 15785 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:02:58.293989 15785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-1/data/instance:
uuid: "dda02e20590d42d594496e9eeada25e6"
format_stamp: "Formatted at 2026-08-11 12:02:58 on dist-test-slave-mjjr"
server_key: "17e14fad39ccb2275d1521a542f950d1"
server_key_iv: "1e64cd563c8f6ef20970141c776145d1"
server_key_version: "encryptionkey@0"
I20260811 12:02:58.294764 15785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance:
uuid: "dda02e20590d42d594496e9eeada25e6"
format_stamp: "Formatted at 2026-08-11 12:02:58 on dist-test-slave-mjjr"
server_key: "17e14fad39ccb2275d1521a542f950d1"
server_key_iv: "1e64cd563c8f6ef20970141c776145d1"
server_key_version: "encryptionkey@0"
I20260811 12:02:58.299928 15785 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 12:02:58.303287 15812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:58.304600 15785 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:02:58.304778 15785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-1/wal
uuid: "dda02e20590d42d594496e9eeada25e6"
format_stamp: "Formatted at 2026-08-11 12:02:58 on dist-test-slave-mjjr"
server_key: "17e14fad39ccb2275d1521a542f950d1"
server_key_iv: "1e64cd563c8f6ef20970141c776145d1"
server_key_version: "encryptionkey@0"
I20260811 12:02:58.304980 15785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:02:58.322913 15785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:02:58.327097 15785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:02:58.327389 15785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:02:58.328323 15785 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:02:58.329710 15785 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:02:58.329814 15785 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:58.329962 15785 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:02:58.330039 15785 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:58.363032 15785 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.130:40399
I20260811 12:02:58.363081 15998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.130:40399 every 8 connection(s)
I20260811 12:02:58.364419 15785 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
I20260811 12:02:58.369277 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 15785
I20260811 12:02:58.369493 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance
I20260811 12:02:58.369962 10102 external_mini_cluster.cc:1428] Setting key 3dcb658713e6980d773f0b8f68d37afb
Aug 11 12:02:58 dist-test-slave-mjjr krb5kdc[15395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449778, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.130@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:02:58.389001 16002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:58.366994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:50773 (local address 127.9.221.190:33845)
0811 12:02:58.367438 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:58.367442 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:58.368480 (+ 1038us) server_negotiation.cc:407] Connection header received
0811 12:02:58.370535 (+ 2055us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:58.370540 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:58.370627 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:58.370746 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:58.372776 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:58.373557 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:58.376071 (+ 2514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:58.376823 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:58.376959 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:58.381580 (+ 4621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:58.381590 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:58.381598 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:58.381645 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:58.385510 (+ 3865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:58.386231 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:58.386240 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:58.386246 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:58.386418 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:58.386845 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:58.386852 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:58.386858 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:58.387273 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:58.387428 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:58.388196 (+ 768us) server_negotiation.cc:299] Negotiation successful
0811 12:02:58.388479 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":219,"threads_started":1}
I20260811 12:02:58.389590 16003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:58.367563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:33845 (local address 127.9.221.130:50773)
0811 12:02:58.368310 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:58.368347 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:58.370264 (+ 1917us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:58.371042 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:58.371054 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:58.371861 (+ 807us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:02:58.372529 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:58.372547 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:58.373727 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:58.373736 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:58.375920 (+ 2184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:58.375938 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:58.377193 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:58.377202 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:58.377541 (+ 339us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:58.378359 (+ 818us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:02:58.378400 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:02:58.381334 (+ 2934us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:02:58.385648 (+ 4314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:58.385655 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:58.385668 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:58.386085 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:58.386546 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:58.386549 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:58.386551 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:58.386726 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:58.387395 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:02:58.387402 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:02:58.387805 (+ 403us) client_negotiation.cc:769] Sending connection context
0811 12:02:58.388034 (+ 229us) client_negotiation.cc:240] Negotiation successful
0811 12:02:58.388444 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":281,"threads_started":1}
I20260811 12:02:58.391185 16000 heartbeater.cc:347] Connected to a master server at 127.9.221.190:33845
I20260811 12:02:58.391549 16000 heartbeater.cc:464] Registering TS with master...
I20260811 12:02:58.392582 16000 heartbeater.cc:511] Master 127.9.221.190:33845 requested a full tablet report, sending...
I20260811 12:02:58.394343 15476 ts_manager.cc:194] Registered new tserver with Master: dda02e20590d42d594496e9eeada25e6 (127.9.221.130:40399)
WARNING: no policy specified for kudu/127.9.221.131@KRBTEST.COM; defaulting to no policy
I20260811 12:02:58.395404 15476 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.130@KRBTEST.COM'} at 127.9.221.130:50773
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab.
Entry for principal kudu/127.9.221.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab.
I20260811 12:02:58.421049 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.131:0
--local_ip_for_outbound_sockets=127.9.221.131
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:33845
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab
--principal=kudu/127.9.221.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:34627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:02:58.760156 16006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:02:58.760442 16006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:02:58.760633 16006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:02:58.768083 16006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:02:58.768211 16006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:02:58.768394 16006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.131
I20260811 12:02:58.779461 16006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:34627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab
--principal=kudu/127.9.221.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:33845
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.131
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:02:58.780871 16006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:02:58.782166 16006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:02:58.795253 16013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:58.795215 16014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:02:58.795374 16018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:02:58.796531 16006 server_base.cc:1034] running on GCE node
I20260811 12:02:58.797070 16006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:02:58.797958 16006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:02:58.799079 15774 heartbeater.cc:503] Master 127.9.221.190:33845 was elected leader, sending a full tablet report...
I20260811 12:02:58.799172 16006 hybrid_clock.cc:648] HybridClock initialized: now 1786449778799102 us; error 45 us; skew 500 ppm
I20260811 12:02:58.799467 16006 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:02:58 dist-test-slave-mjjr krb5kdc[15395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449778, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:02:58.807124 16006 init.cc:377] Logged in from keytab as kudu/127.9.221.131@KRBTEST.COM (short username kudu)
I20260811 12:02:58.808732 16006 webserver.cc:460] Webserver started at http://127.9.221.131:43931/ using document root <none> and password file <none>
I20260811 12:02:58.809525 16006 fs_manager.cc:359] Metadata directory not provided
I20260811 12:02:58.809671 16006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:02:58.809999 16006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:02:58.812942 16006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-2/data/instance:
uuid: "b8be87273ae14a3a90cdaadeef757055"
format_stamp: "Formatted at 2026-08-11 12:02:58 on dist-test-slave-mjjr"
server_key: "e752373cfdb198dc6ff72c42397f1877"
server_key_iv: "f8236430fa253a73066bf519fb76bb3d"
server_key_version: "encryptionkey@0"
I20260811 12:02:58.813704 16006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance:
uuid: "b8be87273ae14a3a90cdaadeef757055"
format_stamp: "Formatted at 2026-08-11 12:02:58 on dist-test-slave-mjjr"
server_key: "e752373cfdb198dc6ff72c42397f1877"
server_key_iv: "f8236430fa253a73066bf519fb76bb3d"
server_key_version: "encryptionkey@0"
I20260811 12:02:58.818944 16006 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 12:02:58.822266 16028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:58.823561 16006 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:02:58.823762 16006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-2/wal
uuid: "b8be87273ae14a3a90cdaadeef757055"
format_stamp: "Formatted at 2026-08-11 12:02:58 on dist-test-slave-mjjr"
server_key: "e752373cfdb198dc6ff72c42397f1877"
server_key_iv: "f8236430fa253a73066bf519fb76bb3d"
server_key_version: "encryptionkey@0"
I20260811 12:02:58.823971 16006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:02:58.840417 16006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:02:58.844594 16006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:02:58.844883 16006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:02:58.845768 16006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:02:58.847069 16006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:02:58.847173 16006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:58.847318 16006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:02:58.847391 16006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:02:58.880213 16006 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.131:33039
I20260811 12:02:58.880264 16210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.131:33039 every 8 connection(s)
I20260811 12:02:58.881487 16006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
I20260811 12:02:58.881832 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 16006
I20260811 12:02:58.882038 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance
I20260811 12:02:58.882671 10102 external_mini_cluster.cc:1428] Setting key cd781d16d79bb2f645dd06681355325d
Aug 11 12:02:58 dist-test-slave-mjjr krb5kdc[15395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449778, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.131@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:02:58.901450 16002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:58.884113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:35683 (local address 127.9.221.190:33845)
0811 12:02:58.884338 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:58.884344 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:58.885804 (+ 1460us) server_negotiation.cc:407] Connection header received
0811 12:02:58.887688 (+ 1884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:58.887694 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:58.887799 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:58.887911 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:58.889339 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:58.890081 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:58.891636 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:58.892075 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:58.892218 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:58.895470 (+ 3252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:58.895475 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:58.895478 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:58.895510 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:58.898496 (+ 2986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:58.899185 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:58.899192 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:58.899199 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:58.899324 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:58.899799 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:58.899805 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:58.899808 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:58.900083 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:58.900240 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:58.900864 (+ 624us) server_negotiation.cc:299] Negotiation successful
0811 12:02:58.901127 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 12:02:58.902354 16215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:58.884789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:33845 (local address 127.9.221.131:35683)
0811 12:02:58.885653 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0811 12:02:58.885699 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 12:02:58.887400 (+ 1701us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:02:58.888148 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:02:58.888156 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:02:58.888644 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:02:58.889174 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:58.889190 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:58.890251 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:58.890255 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:58.891490 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:02:58.891503 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:02:58.892313 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:02:58.892320 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:02:58.892538 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:58.893121 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:02:58.893146 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:02:58.895265 (+ 2119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:02:58.898641 (+ 3376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:58.898649 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:58.898662 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:58.899061 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:58.899451 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:02:58.899456 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:02:58.899465 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:02:58.899630 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:02:58.900221 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:02:58.900228 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:02:58.900535 (+ 307us) client_negotiation.cc:769] Sending connection context
0811 12:02:58.900792 (+ 257us) client_negotiation.cc:240] Negotiation successful
0811 12:02:58.901294 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":356,"threads_started":1}
I20260811 12:02:58.903510 16212 heartbeater.cc:347] Connected to a master server at 127.9.221.190:33845
I20260811 12:02:58.903805 16212 heartbeater.cc:464] Registering TS with master...
I20260811 12:02:58.904507 16212 heartbeater.cc:511] Master 127.9.221.190:33845 requested a full tablet report, sending...
I20260811 12:02:58.905826 15476 ts_manager.cc:194] Registered new tserver with Master: b8be87273ae14a3a90cdaadeef757055 (127.9.221.131:33039)
I20260811 12:02:58.906668 15476 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.131@KRBTEST.COM'} at 127.9.221.131:35683
I20260811 12:02:58.910096 10102 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 12:02:58 dist-test-slave-mjjr krb5kdc[15395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449778, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 11 12:02:58 dist-test-slave-mjjr krb5kdc[15395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449778, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:02:58.962807 16002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:58.948714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55382 (local address 127.9.221.190:33845)
0811 12:02:58.948962 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:58.948970 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:58.949311 (+ 341us) server_negotiation.cc:407] Connection header received
0811 12:02:58.949577 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:58.949582 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:58.949659 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:58.949772 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:58.951211 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:58.952054 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:58.953304 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:58.953728 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:58.953868 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:58.957619 (+ 3751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:58.957624 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:58.957627 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:58.957660 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:58.960116 (+ 2456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:58.960724 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:58.960730 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:58.960734 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:58.960875 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:58.961271 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:58.961277 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:58.961282 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:58.961625 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:58.961921 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:58.962228 (+ 307us) server_negotiation.cc:299] Negotiation successful
0811 12:02:58.962468 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260811 12:02:58.963343 15476 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:55382
I20260811 12:02:58.965431 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 15569
Aug 11 12:02:58 dist-test-slave-mjjr krb5kdc[15395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449778, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.9.221.129@KRBTEST.COM
I20260811 12:02:58.980829 16229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:58.966352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60466 (local address 127.9.221.129:32877)
0811 12:02:58.966977 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:58.966984 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:58.967009 (+ 25us) server_negotiation.cc:407] Connection header received
0811 12:02:58.967076 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:58.967082 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:58.967277 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:58.967416 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:58.968571 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:58.969564 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:58.970656 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:58.971205 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:58.971304 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:58.975226 (+ 3922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:58.975238 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:58.975251 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:58.975282 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:58.978039 (+ 2757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:58.978695 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:58.978702 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:58.978704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:58.978822 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:58.979214 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:58.979220 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:58.979223 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:58.979576 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:58.979909 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:58.980212 (+ 303us) server_negotiation.cc:299] Negotiation successful
0811 12:02:58.980485 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":296,"threads_started":1}
W20260811 12:02:58.982043 15768 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:60466
W20260811 12:02:58.983109 10102 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 15569: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:02:58.983286 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 15569
I20260811 12:02:59.009058 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 15785
Aug 11 12:02:59 dist-test-slave-mjjr krb5kdc[15395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449778, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.9.221.130@KRBTEST.COM
I20260811 12:02:59.024163 16230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:59.009916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49522 (local address 127.9.221.130:40399)
0811 12:02:59.010531 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:59.010538 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:59.010564 (+ 26us) server_negotiation.cc:407] Connection header received
0811 12:02:59.010626 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:59.010632 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:59.010840 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:59.010973 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:59.012218 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:59.013211 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:59.014382 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:59.014924 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:59.015020 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:59.019056 (+ 4036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:59.019067 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:59.019081 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:59.019113 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:59.021799 (+ 2686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:59.022376 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:59.022382 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:59.022385 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:59.022517 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:59.022806 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:59.022809 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:59.022811 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:59.023116 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:59.023345 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:59.023567 (+ 222us) server_negotiation.cc:299] Negotiation successful
0811 12:02:59.023867 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":316,"threads_started":1}
W20260811 12:02:59.025626 15994 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:49522
W20260811 12:02:59.026649 10102 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 15785: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:02:59.026786 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 15785
I20260811 12:02:59.051091 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 16006
Aug 11 12:02:59 dist-test-slave-mjjr krb5kdc[15395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449778, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.9.221.131@KRBTEST.COM
I20260811 12:02:59.071214 16231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:02:59.052021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43318 (local address 127.9.221.131:33039)
0811 12:02:59.052664 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0811 12:02:59.052671 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:02:59.052700 (+ 29us) server_negotiation.cc:407] Connection header received
0811 12:02:59.052762 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:02:59.052768 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:02:59.052970 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:02:59.053102 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:02:59.054372 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:59.055385 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:59.056678 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:02:59.057242 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:02:59.057343 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:02:59.062677 (+ 5334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:02:59.062695 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:02:59.062716 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:02:59.062762 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:02:59.068136 (+ 5374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:59.068939 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:59.068947 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:59.068949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:59.069097 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:02:59.069504 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:02:59.069509 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:02:59.069514 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:02:59.069882 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:02:59.070235 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0811 12:02:59.070550 (+ 315us) server_negotiation.cc:299] Negotiation successful
0811 12:02:59.070837 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":313,"threads_started":1}
W20260811 12:02:59.072751 16202 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:43318
W20260811 12:02:59.073902 10102 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 16006: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:02:59.074126 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 16006
I20260811 12:02:59.099010 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 15418
I20260811 12:02:59.100229 15523 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:02:59.204032 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 15418
2026-08-11T12:02:59Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (5376 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:02:59 dist-test-slave-mjjr krb5kdc[16238](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:02:59 dist-test-slave-mjjr krb5kdc[16238](info): set up 2 sockets
Aug 11 12:02:59 dist-test-slave-mjjr krb5kdc[16238](info): commencing operation
krb5kdc: starting...
W20260811 12:03:00.497164 10102 mini_kdc.cc:121] Time spent starting KDC: real 1.245s user 0.002s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Aug 11 12:03:00 dist-test-slave-mjjr krb5kdc[16238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:03:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:03:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.221.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
I20260811 12:03:00.698540 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:44547
--webserver_interface=127.9.221.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:44061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:44547
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:03:01.025740 16260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:03:01.026105 16260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:03:01.026270 16260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:03:01.033215 16260 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:03:01.033361 16260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:03:01.033470 16260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:03:01.033584 16260 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:03:01.033676 16260 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:03:01.042771 16260 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:44061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:44547
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:44547
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.221.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:03:01.044054 16260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:03:01.045342 16260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:03:01.056638 16277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:01.056699 16272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:01.056725 16270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:03:01.057883 16260 server_base.cc:1034] running on GCE node
I20260811 12:03:01.058596 16260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:03:01.059962 16260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:03:01.061163 16260 hybrid_clock.cc:648] HybridClock initialized: now 1786449781061123 us; error 33 us; skew 500 ppm
I20260811 12:03:01.061501 16260 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:03:01 dist-test-slave-mjjr krb5kdc[16238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449781, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:03:01.064553 16260 init.cc:377] Logged in from keytab as kudu/127.9.221.190@KRBTEST.COM (short username kudu)
I20260811 12:03:01.065971 16260 webserver.cc:460] Webserver started at http://127.9.221.190:42121/ using document root <none> and password file <none>
I20260811 12:03:01.066632 16260 fs_manager.cc:359] Metadata directory not provided
I20260811 12:03:01.066778 16260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:03:01.067090 16260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:03:01.069947 16260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "fdbc73e14fa148ddb1c2d26a080d0f6f"
format_stamp: "Formatted at 2026-08-11 12:03:01 on dist-test-slave-mjjr"
server_key: "ae0fff32fd904f66b15d02b5461c086c"
server_key_iv: "065faae3118edae375351fe242b2f2b2"
server_key_version: "encryptionkey@0"
I20260811 12:03:01.070708 16260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "fdbc73e14fa148ddb1c2d26a080d0f6f"
format_stamp: "Formatted at 2026-08-11 12:03:01 on dist-test-slave-mjjr"
server_key: "ae0fff32fd904f66b15d02b5461c086c"
server_key_iv: "065faae3118edae375351fe242b2f2b2"
server_key_version: "encryptionkey@0"
I20260811 12:03:01.075443 16260 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.005s
I20260811 12:03:01.078794 16290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:01.080134 16260 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:03:01.080339 16260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "fdbc73e14fa148ddb1c2d26a080d0f6f"
format_stamp: "Formatted at 2026-08-11 12:03:01 on dist-test-slave-mjjr"
server_key: "ae0fff32fd904f66b15d02b5461c086c"
server_key_iv: "065faae3118edae375351fe242b2f2b2"
server_key_version: "encryptionkey@0"
I20260811 12:03:01.080538 16260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:03:01.121692 16260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:03:01.125885 16260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:03:01.126176 16260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:03:01.144824 16260 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.190:44547
I20260811 12:03:01.144896 16373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.190:44547 every 8 connection(s)
I20260811 12:03:01.146219 16260 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:03:01.150015 16377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:03:01.155359 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 16260
I20260811 12:03:01.155592 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:03:01.156415 10102 external_mini_cluster.cc:1428] Setting key 8425d518d7ba654c9b77289f6c362246
I20260811 12:03:01.162071 16377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdbc73e14fa148ddb1c2d26a080d0f6f: Bootstrap starting.
I20260811 12:03:01.165220 16377 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fdbc73e14fa148ddb1c2d26a080d0f6f: Neither blocks nor log segments found. Creating new log.
I20260811 12:03:01.166287 16377 log.cc:824] T 00000000000000000000000000000000 P fdbc73e14fa148ddb1c2d26a080d0f6f: Log is configured to *not* fsync() on all Append() calls
I20260811 12:03:01.169291 16377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdbc73e14fa148ddb1c2d26a080d0f6f: No bootstrap required, opened a new log
Aug 11 12:03:01 dist-test-slave-mjjr krb5kdc[16238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:03:01.179124 16382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:01.158720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55280 (local address 127.9.221.190:44547)
0811 12:03:01.159552 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:01.159567 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:01.159610 (+ 43us) server_negotiation.cc:407] Connection header received
0811 12:03:01.161423 (+ 1813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:01.161448 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:01.161805 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:01.162188 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:01.163743 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:01.165032 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:01.166318 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:01.167099 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:01.167301 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:01.171757 (+ 4456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:01.171766 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:01.171779 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:01.171812 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:01.174914 (+ 3102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:01.175776 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:01.175785 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:01.175790 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:01.175963 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:01.176372 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:01.176375 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:01.176377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:01.177010 (+ 633us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:01.177218 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:01.177714 (+ 496us) server_negotiation.cc:299] Negotiation successful
0811 12:03:01.178095 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":612,"thread_start_us":391,"threads_started":1}
I20260811 12:03:01.179071 16377 raft_consensus.cc:348] T 00000000000000000000000000000000 P fdbc73e14fa148ddb1c2d26a080d0f6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdbc73e14fa148ddb1c2d26a080d0f6f" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 44547 } }
I20260811 12:03:01.179549 16377 raft_consensus.cc:374] T 00000000000000000000000000000000 P fdbc73e14fa148ddb1c2d26a080d0f6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:03:01.179708 16377 raft_consensus.cc:729] T 00000000000000000000000000000000 P fdbc73e14fa148ddb1c2d26a080d0f6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdbc73e14fa148ddb1c2d26a080d0f6f, State: Initialized, Role: FOLLOWER
I20260811 12:03:01.180291 16377 consensus_queue.cc:260] T 00000000000000000000000000000000 P fdbc73e14fa148ddb1c2d26a080d0f6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdbc73e14fa148ddb1c2d26a080d0f6f" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 44547 } }
I20260811 12:03:01.180579 16377 raft_consensus.cc:388] T 00000000000000000000000000000000 P fdbc73e14fa148ddb1c2d26a080d0f6f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:03:01.180756 16377 raft_consensus.cc:482] T 00000000000000000000000000000000 P fdbc73e14fa148ddb1c2d26a080d0f6f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:03:01.180948 16377 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fdbc73e14fa148ddb1c2d26a080d0f6f [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:03:01.183280 16377 raft_consensus.cc:504] T 00000000000000000000000000000000 P fdbc73e14fa148ddb1c2d26a080d0f6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdbc73e14fa148ddb1c2d26a080d0f6f" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 44547 } }
I20260811 12:03:01.183882 16377 leader_election.cc:302] T 00000000000000000000000000000000 P fdbc73e14fa148ddb1c2d26a080d0f6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fdbc73e14fa148ddb1c2d26a080d0f6f; no voters:
I20260811 12:03:01.184580 16377 leader_election.cc:288] T 00000000000000000000000000000000 P fdbc73e14fa148ddb1c2d26a080d0f6f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:03:01.184661 16385 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fdbc73e14fa148ddb1c2d26a080d0f6f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:03:01.185393 16385 raft_consensus.cc:686] T 00000000000000000000000000000000 P fdbc73e14fa148ddb1c2d26a080d0f6f [term 1 LEADER]: Becoming Leader. State: Replica: fdbc73e14fa148ddb1c2d26a080d0f6f, State: Running, Role: LEADER
I20260811 12:03:01.186004 16385 consensus_queue.cc:237] T 00000000000000000000000000000000 P fdbc73e14fa148ddb1c2d26a080d0f6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdbc73e14fa148ddb1c2d26a080d0f6f" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 44547 } }
I20260811 12:03:01.186805 16377 sys_catalog.cc:564] T 00000000000000000000000000000000 P fdbc73e14fa148ddb1c2d26a080d0f6f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:03:01.189236 16387 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdbc73e14fa148ddb1c2d26a080d0f6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fdbc73e14fa148ddb1c2d26a080d0f6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdbc73e14fa148ddb1c2d26a080d0f6f" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 44547 } } }
I20260811 12:03:01.189735 16387 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdbc73e14fa148ddb1c2d26a080d0f6f [sys.catalog]: This master's current role is: LEADER
I20260811 12:03:01.190524 16388 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdbc73e14fa148ddb1c2d26a080d0f6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader fdbc73e14fa148ddb1c2d26a080d0f6f. Latest consensus state: current_term: 1 leader_uuid: "fdbc73e14fa148ddb1c2d26a080d0f6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdbc73e14fa148ddb1c2d26a080d0f6f" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 44547 } } }
I20260811 12:03:01.190977 16388 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdbc73e14fa148ddb1c2d26a080d0f6f [sys.catalog]: This master's current role is: LEADER
I20260811 12:03:01.191475 16394 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:03:01.198101 16394 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:03:01.206713 16394 catalog_manager.cc:1347] Generated new cluster ID: be6c27cf59344291a747fad5efcd7106
I20260811 12:03:01.206810 16394 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:03:01.231168 16394 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:03:01.232506 16394 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:03:01.243000 16394 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fdbc73e14fa148ddb1c2d26a080d0f6f: Generated new TSK 0
I20260811 12:03:01.243801 16394 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.221.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab.
Entry for principal kudu/127.9.221.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab.
I20260811 12:03:01.303165 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.129:0
--local_ip_for_outbound_sockets=127.9.221.129
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:44547
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab
--principal=kudu/127.9.221.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:44061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:03:01.644923 16415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:03:01.645175 16415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:03:01.645265 16415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:03:01.652324 16415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:03:01.652438 16415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:03:01.652630 16415 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.129
I20260811 12:03:01.661998 16415 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:44061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab
--principal=kudu/127.9.221.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:44547
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.129
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:03:01.663232 16415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:03:01.664561 16415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:03:01.677443 16422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:01.677445 16425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:01.677702 16427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:03:01.678355 16415 server_base.cc:1034] running on GCE node
I20260811 12:03:01.678999 16415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:03:01.680104 16415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:03:01.681335 16415 hybrid_clock.cc:648] HybridClock initialized: now 1786449781681301 us; error 49 us; skew 500 ppm
I20260811 12:03:01.681639 16415 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:03:01 dist-test-slave-mjjr krb5kdc[16238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449781, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:03:01.684855 16415 init.cc:377] Logged in from keytab as kudu/127.9.221.129@KRBTEST.COM (short username kudu)
I20260811 12:03:01.686262 16415 webserver.cc:460] Webserver started at http://127.9.221.129:33705/ using document root <none> and password file <none>
I20260811 12:03:01.687018 16415 fs_manager.cc:359] Metadata directory not provided
I20260811 12:03:01.687166 16415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:03:01.687491 16415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:03:01.690351 16415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-0/data/instance:
uuid: "494001a0ca64472f97e436b758f2ab35"
format_stamp: "Formatted at 2026-08-11 12:03:01 on dist-test-slave-mjjr"
server_key: "bbe817feb2ed4f727945e24ff31e8aa0"
server_key_iv: "306950e785d7b4da968c034e53d81f28"
server_key_version: "encryptionkey@0"
I20260811 12:03:01.691188 16415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance:
uuid: "494001a0ca64472f97e436b758f2ab35"
format_stamp: "Formatted at 2026-08-11 12:03:01 on dist-test-slave-mjjr"
server_key: "bbe817feb2ed4f727945e24ff31e8aa0"
server_key_iv: "306950e785d7b4da968c034e53d81f28"
server_key_version: "encryptionkey@0"
I20260811 12:03:01.696558 16415 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 12:03:01.699963 16438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:01.701182 16415 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 12:03:01.701355 16415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-0/wal
uuid: "494001a0ca64472f97e436b758f2ab35"
format_stamp: "Formatted at 2026-08-11 12:03:01 on dist-test-slave-mjjr"
server_key: "bbe817feb2ed4f727945e24ff31e8aa0"
server_key_iv: "306950e785d7b4da968c034e53d81f28"
server_key_version: "encryptionkey@0"
I20260811 12:03:01.701601 16415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:03:01.724210 16415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:03:01.727788 16415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:03:01.728017 16415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:03:01.728842 16415 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:03:01.730073 16415 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:03:01.730145 16415 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:01.730225 16415 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:03:01.730278 16415 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:01.766652 16415 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.129:43057
I20260811 12:03:01.766739 16618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.129:43057 every 8 connection(s)
I20260811 12:03:01.768275 16415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
I20260811 12:03:01.775941 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 16415
I20260811 12:03:01.776132 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance
I20260811 12:03:01.776772 10102 external_mini_cluster.cc:1428] Setting key 91c23dd498c76558536fc865d934a08a
Aug 11 12:03:01 dist-test-slave-mjjr krb5kdc[16238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449781, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:03:01.789713 16625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:01.770852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:42601 (local address 127.9.221.190:44547)
0811 12:03:01.771300 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:01.771305 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:01.772313 (+ 1008us) server_negotiation.cc:407] Connection header received
0811 12:03:01.774165 (+ 1852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:01.774170 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:01.774261 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:01.774384 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:01.775881 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:01.776740 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:01.778796 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:01.779405 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:01.779562 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:01.782935 (+ 3373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:01.782943 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:01.782951 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:01.782995 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:01.786594 (+ 3599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:01.787415 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:01.787419 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:01.787422 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:01.787533 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:01.788001 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:01.788005 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:01.788010 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:01.788277 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:01.788434 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:01.789115 (+ 681us) server_negotiation.cc:299] Negotiation successful
0811 12:03:01.789344 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":236,"threads_started":1}
I20260811 12:03:01.790598 16626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:01.771409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44547 (local address 127.9.221.129:42601)
0811 12:03:01.772149 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:01.772192 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:01.773900 (+ 1708us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:01.774616 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:01.774623 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:01.775106 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:01.775718 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:01.775736 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:01.776903 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:01.776909 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:01.778660 (+ 1751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:01.778676 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:01.779634 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:01.779679 (+ 45us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:01.779887 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:01.780592 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:01.780614 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:01.782731 (+ 2117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:01.786739 (+ 4008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:01.786747 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:01.786763 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:01.787262 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:01.787667 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:01.787672 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:01.787676 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:01.787897 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:01.788420 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:01.788428 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:01.788739 (+ 311us) client_negotiation.cc:769] Sending connection context
0811 12:03:01.788986 (+ 247us) client_negotiation.cc:240] Negotiation successful
0811 12:03:01.789495 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":334,"threads_started":1}
I20260811 12:03:01.792120 16620 heartbeater.cc:347] Connected to a master server at 127.9.221.190:44547
I20260811 12:03:01.792418 16620 heartbeater.cc:464] Registering TS with master...
I20260811 12:03:01.793092 16620 heartbeater.cc:511] Master 127.9.221.190:44547 requested a full tablet report, sending...
I20260811 12:03:01.795113 16314 ts_manager.cc:194] Registered new tserver with Master: 494001a0ca64472f97e436b758f2ab35 (127.9.221.129:43057)
I20260811 12:03:01.796634 16314 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.129@KRBTEST.COM'} at 127.9.221.129:42601
WARNING: no policy specified for kudu/127.9.221.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab.
Entry for principal kudu/127.9.221.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab.
I20260811 12:03:01.824015 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.130:0
--local_ip_for_outbound_sockets=127.9.221.130
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:44547
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab
--principal=kudu/127.9.221.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:44061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:03:02.155160 16632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:03:02.155475 16632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:03:02.155691 16632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:03:02.163043 16632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:03:02.163192 16632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:03:02.163362 16632 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.130
I20260811 12:03:02.173017 16632 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:44061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab
--principal=kudu/127.9.221.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:44547
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.130
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:03:02.174299 16632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:03:02.175587 16632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:03:02.188674 16638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:02.188669 16640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:02.188974 16637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:03:02.189292 16632 server_base.cc:1034] running on GCE node
I20260811 12:03:02.189869 16632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:03:02.190739 16632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:03:02.191933 16632 hybrid_clock.cc:648] HybridClock initialized: now 1786449782191859 us; error 56 us; skew 500 ppm
I20260811 12:03:02.192242 16632 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:03:02 dist-test-slave-mjjr krb5kdc[16238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449782, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:03:02.195303 16632 init.cc:377] Logged in from keytab as kudu/127.9.221.130@KRBTEST.COM (short username kudu)
I20260811 12:03:02.196707 16632 webserver.cc:460] Webserver started at http://127.9.221.130:35769/ using document root <none> and password file <none>
I20260811 12:03:02.197362 16632 fs_manager.cc:359] Metadata directory not provided
I20260811 12:03:02.197463 16632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:03:02.197705 16632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:03:02.200495 16632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-1/data/instance:
uuid: "bb76df19dbcf40aaa983dddc07f0f9dd"
format_stamp: "Formatted at 2026-08-11 12:03:02 on dist-test-slave-mjjr"
server_key: "14b5ee9fe449ef6d060fce26967ac560"
server_key_iv: "98dfa09daafdfca08e05017598fc5cdf"
server_key_version: "encryptionkey@0"
I20260811 12:03:02.201196 16632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance:
uuid: "bb76df19dbcf40aaa983dddc07f0f9dd"
format_stamp: "Formatted at 2026-08-11 12:03:02 on dist-test-slave-mjjr"
server_key: "14b5ee9fe449ef6d060fce26967ac560"
server_key_iv: "98dfa09daafdfca08e05017598fc5cdf"
server_key_version: "encryptionkey@0"
I20260811 12:03:02.206095 16632 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 12:03:02.209362 16651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:02.210546 16632 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 12:03:02.210712 16632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-1/wal
uuid: "bb76df19dbcf40aaa983dddc07f0f9dd"
format_stamp: "Formatted at 2026-08-11 12:03:02 on dist-test-slave-mjjr"
server_key: "14b5ee9fe449ef6d060fce26967ac560"
server_key_iv: "98dfa09daafdfca08e05017598fc5cdf"
server_key_version: "encryptionkey@0"
I20260811 12:03:02.210911 16632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:03:02.235364 16632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:03:02.239100 16632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:03:02.239404 16632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:03:02.240396 16632 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:03:02.241663 16632 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:03:02.241768 16632 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:02.241911 16632 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:03:02.241995 16632 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:02.274770 16632 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.130:38155
I20260811 12:03:02.274820 16827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.130:38155 every 8 connection(s)
I20260811 12:03:02.276129 16632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
I20260811 12:03:02.282954 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 16632
I20260811 12:03:02.283142 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance
I20260811 12:03:02.283699 10102 external_mini_cluster.cc:1428] Setting key 3e9fc4b5ce63c5472c25e40cbc50ef4a
Aug 11 12:03:02 dist-test-slave-mjjr krb5kdc[16238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449782, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.130@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:03:02.296635 16625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:02.278770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:53155 (local address 127.9.221.190:44547)
0811 12:03:02.278980 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:02.278986 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:02.280228 (+ 1242us) server_negotiation.cc:407] Connection header received
0811 12:03:02.282091 (+ 1863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:02.282096 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:02.282192 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:02.282305 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:02.283916 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:02.284910 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:02.286559 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:02.287063 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:02.287193 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:02.290867 (+ 3674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:02.290872 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:02.290875 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:02.290905 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:02.293631 (+ 2726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:02.294362 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:02.294368 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:02.294371 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:02.294533 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:02.294951 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:02.294956 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:02.294959 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:02.295226 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:02.295378 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:02.296079 (+ 701us) server_negotiation.cc:299] Negotiation successful
0811 12:03:02.296304 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 12:03:02.297626 16831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:02.279272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44547 (local address 127.9.221.130:53155)
0811 12:03:02.280015 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:02.280086 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:02.281821 (+ 1735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:02.282536 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:02.282543 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:02.283017 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:02.283726 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:02.283745 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:02.285112 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:02.285115 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:02.286383 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:02.286395 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:02.287274 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:02.287281 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:02.287512 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:02.288291 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:02.288313 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:02.290646 (+ 2333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:02.293772 (+ 3126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:02.293800 (+ 28us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:02.293817 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:02.294238 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:02.294633 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:02.294637 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:02.294641 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:02.294840 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:02.295357 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:02.295365 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:02.295716 (+ 351us) client_negotiation.cc:769] Sending connection context
0811 12:03:02.295969 (+ 253us) client_negotiation.cc:240] Negotiation successful
0811 12:03:02.296457 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":316,"threads_started":1}
I20260811 12:03:02.299126 16829 heartbeater.cc:347] Connected to a master server at 127.9.221.190:44547
I20260811 12:03:02.299417 16829 heartbeater.cc:464] Registering TS with master...
I20260811 12:03:02.300146 16829 heartbeater.cc:511] Master 127.9.221.190:44547 requested a full tablet report, sending...
I20260811 12:03:02.301539 16314 ts_manager.cc:194] Registered new tserver with Master: bb76df19dbcf40aaa983dddc07f0f9dd (127.9.221.130:38155)
I20260811 12:03:02.302323 16314 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.130@KRBTEST.COM'} at 127.9.221.130:53155
WARNING: no policy specified for kudu/127.9.221.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab.
Entry for principal kudu/127.9.221.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab.
I20260811 12:03:02.328541 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.131:0
--local_ip_for_outbound_sockets=127.9.221.131
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:44547
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab
--principal=kudu/127.9.221.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:44061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:03:02.656647 16835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:03:02.656972 16835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:03:02.657140 16835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:03:02.664359 16835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:03:02.664500 16835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:03:02.664676 16835 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.131
I20260811 12:03:02.674098 16835 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:44061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab
--principal=kudu/127.9.221.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:44547
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.131
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:03:02.675336 16835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:03:02.676587 16835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:03:02.689659 16842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:02.689627 16843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:02.689893 16845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:03:02.690402 16835 server_base.cc:1034] running on GCE node
I20260811 12:03:02.690896 16835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:03:02.691794 16835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:03:02.693019 16835 hybrid_clock.cc:648] HybridClock initialized: now 1786449782692965 us; error 43 us; skew 500 ppm
I20260811 12:03:02.693367 16835 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:03:02 dist-test-slave-mjjr krb5kdc[16238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449782, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:03:02.696480 16835 init.cc:377] Logged in from keytab as kudu/127.9.221.131@KRBTEST.COM (short username kudu)
I20260811 12:03:02.697770 16835 webserver.cc:460] Webserver started at http://127.9.221.131:41447/ using document root <none> and password file <none>
I20260811 12:03:02.698371 16835 fs_manager.cc:359] Metadata directory not provided
I20260811 12:03:02.698469 16835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:03:02.698767 16835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:03:02.701593 16835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-2/data/instance:
uuid: "bea5092460db44419572a9ed873ba2c6"
format_stamp: "Formatted at 2026-08-11 12:03:02 on dist-test-slave-mjjr"
server_key: "937a91c039881055f8362ebdefbfcef5"
server_key_iv: "cf82bd6ad897c0c3a0dd3f08b02246ea"
server_key_version: "encryptionkey@0"
I20260811 12:03:02.702297 16835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance:
uuid: "bea5092460db44419572a9ed873ba2c6"
format_stamp: "Formatted at 2026-08-11 12:03:02 on dist-test-slave-mjjr"
server_key: "937a91c039881055f8362ebdefbfcef5"
server_key_iv: "cf82bd6ad897c0c3a0dd3f08b02246ea"
server_key_version: "encryptionkey@0"
I20260811 12:03:02.707221 16835 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260811 12:03:02.710455 16854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:02.711715 16835 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 12:03:02.711853 16835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-2/wal
uuid: "bea5092460db44419572a9ed873ba2c6"
format_stamp: "Formatted at 2026-08-11 12:03:02 on dist-test-slave-mjjr"
server_key: "937a91c039881055f8362ebdefbfcef5"
server_key_iv: "cf82bd6ad897c0c3a0dd3f08b02246ea"
server_key_version: "encryptionkey@0"
I20260811 12:03:02.712000 16835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:03:02.727929 16835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:03:02.732060 16835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:03:02.732280 16835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:03:02.733117 16835 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:03:02.734380 16835 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:03:02.734449 16835 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:02.734531 16835 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:03:02.734584 16835 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:02.767918 16835 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.131:42661
I20260811 12:03:02.767921 17033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.131:42661 every 8 connection(s)
I20260811 12:03:02.769544 16835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
I20260811 12:03:02.776507 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 16835
I20260811 12:03:02.776705 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance
I20260811 12:03:02.777252 10102 external_mini_cluster.cc:1428] Setting key b950bbea13a23a7fd21c0497c595e4df
Aug 11 12:03:02 dist-test-slave-mjjr krb5kdc[16238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449782, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.131@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:03:02.790112 16625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:02.772226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:47945 (local address 127.9.221.190:44547)
0811 12:03:02.772400 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:02.772405 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:02.773714 (+ 1309us) server_negotiation.cc:407] Connection header received
0811 12:03:02.775394 (+ 1680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:02.775399 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:02.775483 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:02.775572 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:02.777543 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:02.778758 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:02.780784 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:02.781316 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:02.781411 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:02.784935 (+ 3524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:02.784940 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:02.784943 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:02.784984 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:02.787524 (+ 2540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:02.788261 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:02.788265 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:02.788267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:02.788393 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:02.788750 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:02.788754 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:02.788755 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:02.789011 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:02.789117 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:02.789703 (+ 586us) server_negotiation.cc:299] Negotiation successful
0811 12:03:02.789855 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 12:03:02.791034 17040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:02.772825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44547 (local address 127.9.221.131:47945)
0811 12:03:02.773525 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:02.773562 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:02.775142 (+ 1580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:02.775845 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:02.775852 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:02.776542 (+ 690us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:02.777323 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:02.777350 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:02.778915 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:02.778921 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:02.780639 (+ 1718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:02.780651 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:02.781517 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:02.781522 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:02.781713 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:02.782340 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:02.782364 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:02.784758 (+ 2394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:02.787698 (+ 2940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:02.787705 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:02.787714 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:02.788154 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:02.788483 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:02.788486 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:02.788488 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:02.788657 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:02.789108 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:02.789113 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:02.789407 (+ 294us) client_negotiation.cc:769] Sending connection context
0811 12:03:02.789606 (+ 199us) client_negotiation.cc:240] Negotiation successful
0811 12:03:02.789923 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":262,"threads_started":1}
I20260811 12:03:02.792429 17034 heartbeater.cc:347] Connected to a master server at 127.9.221.190:44547
I20260811 12:03:02.792768 17034 heartbeater.cc:464] Registering TS with master...
I20260811 12:03:02.793491 17034 heartbeater.cc:511] Master 127.9.221.190:44547 requested a full tablet report, sending...
I20260811 12:03:02.794876 16314 ts_manager.cc:194] Registered new tserver with Master: bea5092460db44419572a9ed873ba2c6 (127.9.221.131:42661)
I20260811 12:03:02.795732 16314 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.131@KRBTEST.COM'} at 127.9.221.131:47945
I20260811 12:03:02.799376 16620 heartbeater.cc:503] Master 127.9.221.190:44547 was elected leader, sending a full tablet report...
I20260811 12:03:02.804867 10102 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 12:03:02 dist-test-slave-mjjr krb5kdc[16238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449782, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 12:03:02 dist-test-slave-mjjr krb5kdc[16238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449782, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.129@KRBTEST.COM
I20260811 12:03:02.856443 17046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:02.841622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41314 (local address 127.9.221.129:43057)
0811 12:03:02.842281 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:02.842292 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:02.842327 (+ 35us) server_negotiation.cc:407] Connection header received
0811 12:03:02.842388 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:02.842394 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:02.842614 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:02.842800 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:02.844317 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:02.845342 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:02.846628 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:02.847200 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:02.847348 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:02.851284 (+ 3936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:02.851296 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:02.851309 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:02.851348 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:02.854006 (+ 2658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:02.854598 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:02.854604 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:02.854607 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:02.854793 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:02.855129 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:02.855133 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:02.855135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:02.855463 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:02.855783 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:02.855964 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 12:03:02.856155 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":299,"threads_started":1}
Aug 11 12:03:02 dist-test-slave-mjjr krb5kdc[16238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449782, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:03:02.873538 16625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:02.861055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55284 (local address 127.9.221.190:44547)
0811 12:03:02.861266 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:02.861276 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:02.861386 (+ 110us) server_negotiation.cc:407] Connection header received
0811 12:03:02.861509 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:02.861513 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:02.861575 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:02.861652 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:02.862830 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:02.863535 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:02.864831 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:02.865235 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:02.865325 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:02.869006 (+ 3681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:02.869011 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:02.869014 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:02.869046 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:02.871368 (+ 2322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:02.871983 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:02.871987 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:02.871989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:02.872112 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:02.872420 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:02.872423 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:02.872425 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:02.872733 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:02.872919 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:02.873138 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 12:03:02.873290 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260811 12:03:02.874225 16314 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55284
I20260811 12:03:02.875275 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 16415
I20260811 12:03:02.887589 17046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:02.876140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41320 (local address 127.9.221.129:43057)
0811 12:03:02.876346 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:02.876354 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:02.876603 (+ 249us) server_negotiation.cc:407] Connection header received
0811 12:03:02.876695 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:02.876698 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:02.876789 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:02.876862 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:02.877946 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:02.878581 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:02.879749 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:02.880210 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:02.880319 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:02.882478 (+ 2159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:02.882483 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:02.882486 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:02.882514 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:02.885275 (+ 2761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:02.885929 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:02.885933 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:02.885935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:02.886076 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:02.886423 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:02.886426 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:02.886428 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:02.886670 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:02.886924 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:02.887143 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 12:03:02.887351 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 12:03:02.888278 16613 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:03:02.970105 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 16415
I20260811 12:03:02.997370 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 16632
Aug 11 12:03:03 dist-test-slave-mjjr krb5kdc[16238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449782, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.130@KRBTEST.COM
I20260811 12:03:03.013185 17058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:02.998348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48730 (local address 127.9.221.130:38155)
0811 12:03:02.998885 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:02.998892 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:02.998917 (+ 25us) server_negotiation.cc:407] Connection header received
0811 12:03:02.998987 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:02.998993 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:02.999201 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:02.999379 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:03.000692 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:03.001736 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:03.002949 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:03.003492 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:03.003643 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:03.007867 (+ 4224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:03.007878 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:03.007893 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:03.007925 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:03.010757 (+ 2832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:03.011324 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:03.011330 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:03.011332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:03.011450 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:03.011824 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:03.011827 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:03.011829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:03.012139 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:03.012365 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:03.012579 (+ 214us) server_negotiation.cc:299] Negotiation successful
0811 12:03:03.012811 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":268,"threads_started":1}
I20260811 12:03:03.014446 16820 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:03:03.097221 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 16632
I20260811 12:03:03.121671 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 16835
Aug 11 12:03:03 dist-test-slave-mjjr krb5kdc[16238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449782, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.131@KRBTEST.COM
I20260811 12:03:03.137261 17063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:03.122522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41034 (local address 127.9.221.131:42661)
0811 12:03:03.123156 (+ 634us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:03.123163 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:03.123193 (+ 30us) server_negotiation.cc:407] Connection header received
0811 12:03:03.123255 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:03.123261 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:03.123483 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:03.123621 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:03.124866 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:03.125864 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:03.127101 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:03.127955 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:03.128053 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:03.132309 (+ 4256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:03.132320 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:03.132333 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:03.132363 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:03.134921 (+ 2558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:03.135565 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:03.135572 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:03.135574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:03.135743 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:03.136034 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:03.136037 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:03.136039 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:03.136336 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:03.136580 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:03.136789 (+ 209us) server_negotiation.cc:299] Negotiation successful
0811 12:03:03.136977 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":309,"threads_started":1}
I20260811 12:03:03.138475 17023 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:03:03.220610 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 16835
I20260811 12:03:03.239275 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 16260
I20260811 12:03:03.240531 16368 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:03:03.342509 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 16260
2026-08-11T12:03:03Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4142 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:03:03 dist-test-slave-mjjr krb5kdc[17069](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:03:03 dist-test-slave-mjjr krb5kdc[17069](info): set up 2 sockets
Aug 11 12:03:03 dist-test-slave-mjjr krb5kdc[17069](info): commencing operation
krb5kdc: starting...
W20260811 12:03:04.654686 10102 mini_kdc.cc:121] Time spent starting KDC: real 1.259s user 0.004s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Aug 11 12:03:04 dist-test-slave-mjjr krb5kdc[17069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449784, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:03:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:03:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.221.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
I20260811 12:03:04.862380 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:33393
--webserver_interface=127.9.221.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:37893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:33393
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:03:05.187279 17088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:03:05.187548 17088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:03:05.187774 17088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:03:05.194752 17088 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:03:05.194869 17088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:03:05.194970 17088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:03:05.195075 17088 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:03:05.195119 17088 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:03:05.204418 17088 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:37893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:33393
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:33393
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.221.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:03:05.205559 17088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:03:05.206825 17088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:03:05.218304 17098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:05.218355 17097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:05.218312 17100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:03:05.218950 17088 server_base.cc:1034] running on GCE node
I20260811 12:03:05.219772 17088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:03:05.221055 17088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:03:05.222280 17088 hybrid_clock.cc:648] HybridClock initialized: now 1786449785222244 us; error 55 us; skew 500 ppm
I20260811 12:03:05.222615 17088 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:03:05 dist-test-slave-mjjr krb5kdc[17069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449785, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:03:05.229264 17088 init.cc:377] Logged in from keytab as kudu/127.9.221.190@KRBTEST.COM (short username kudu)
I20260811 12:03:05.230409 17088 fs_manager.cc:359] Metadata directory not provided
I20260811 12:03:05.230508 17088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:03:05.230773 17088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:03:05.233575 17088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "49898b73a5e547319a48b491d977a457"
format_stamp: "Formatted at 2026-08-11 12:03:05 on dist-test-slave-mjjr"
server_key: "5343b322b12a1811c79ca98da061bf2f"
server_key_iv: "a33c34fdfbbb3f0f36ab2be83fea76c4"
server_key_version: "encryptionkey@0"
I20260811 12:03:05.234285 17088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "49898b73a5e547319a48b491d977a457"
format_stamp: "Formatted at 2026-08-11 12:03:05 on dist-test-slave-mjjr"
server_key: "5343b322b12a1811c79ca98da061bf2f"
server_key_iv: "a33c34fdfbbb3f0f36ab2be83fea76c4"
server_key_version: "encryptionkey@0"
I20260811 12:03:05.239074 17088 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260811 12:03:05.242424 17115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:05.243757 17088 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:03:05.243891 17088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "49898b73a5e547319a48b491d977a457"
format_stamp: "Formatted at 2026-08-11 12:03:05 on dist-test-slave-mjjr"
server_key: "5343b322b12a1811c79ca98da061bf2f"
server_key_iv: "a33c34fdfbbb3f0f36ab2be83fea76c4"
server_key_version: "encryptionkey@0"
I20260811 12:03:05.244086 17088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:03:05.267584 17088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:03:05.271929 17088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:03:05.272138 17088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:03:05.290796 17088 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.190:33393
I20260811 12:03:05.290766 17201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.190:33393 every 8 connection(s)
I20260811 12:03:05.291726 17088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:03:05.295436 17202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:03:05.301164 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 17088
I20260811 12:03:05.301380 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:03:05.302033 10102 external_mini_cluster.cc:1428] Setting key 796999089b00323bedb683a78a4b9505
I20260811 12:03:05.305857 17202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49898b73a5e547319a48b491d977a457: Bootstrap starting.
I20260811 12:03:05.309592 17202 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49898b73a5e547319a48b491d977a457: Neither blocks nor log segments found. Creating new log.
I20260811 12:03:05.310637 17202 log.cc:824] T 00000000000000000000000000000000 P 49898b73a5e547319a48b491d977a457: Log is configured to *not* fsync() on all Append() calls
I20260811 12:03:05.313385 17202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49898b73a5e547319a48b491d977a457: No bootstrap required, opened a new log
Aug 11 12:03:05 dist-test-slave-mjjr krb5kdc[17069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449784, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:03:05.321201 17202 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49898b73a5e547319a48b491d977a457 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49898b73a5e547319a48b491d977a457" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 33393 } }
I20260811 12:03:05.321585 17202 raft_consensus.cc:374] T 00000000000000000000000000000000 P 49898b73a5e547319a48b491d977a457 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:03:05.321699 17202 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49898b73a5e547319a48b491d977a457 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49898b73a5e547319a48b491d977a457, State: Initialized, Role: FOLLOWER
I20260811 12:03:05.322402 17202 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49898b73a5e547319a48b491d977a457 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49898b73a5e547319a48b491d977a457" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 33393 } }
I20260811 12:03:05.322717 17202 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49898b73a5e547319a48b491d977a457 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:03:05.322865 17202 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49898b73a5e547319a48b491d977a457 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:03:05.323087 17202 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49898b73a5e547319a48b491d977a457 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:03:05.324851 17210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:05.304043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56986 (local address 127.9.221.190:33393)
0811 12:03:05.304801 (+ 758us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:05.304812 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:05.304849 (+ 37us) server_negotiation.cc:407] Connection header received
0811 12:03:05.306610 (+ 1761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:05.306630 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:05.306972 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:05.307275 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:05.309272 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:05.310672 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:05.312347 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:05.313003 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:05.313201 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:05.317174 (+ 3973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:05.317184 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:05.317197 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:05.317236 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:05.320470 (+ 3234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:05.321146 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:05.321152 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:05.321157 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:05.321341 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:05.321831 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:05.321836 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:05.321840 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:05.322578 (+ 738us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:05.322899 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:05.323345 (+ 446us) server_negotiation.cc:299] Negotiation successful
0811 12:03:05.323797 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":381,"threads_started":1}
I20260811 12:03:05.325801 17202 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49898b73a5e547319a48b491d977a457 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49898b73a5e547319a48b491d977a457" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 33393 } }
I20260811 12:03:05.326300 17202 leader_election.cc:302] T 00000000000000000000000000000000 P 49898b73a5e547319a48b491d977a457 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49898b73a5e547319a48b491d977a457; no voters:
I20260811 12:03:05.326987 17202 leader_election.cc:288] T 00000000000000000000000000000000 P 49898b73a5e547319a48b491d977a457 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:03:05.327104 17212 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49898b73a5e547319a48b491d977a457 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:03:05.328199 17212 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49898b73a5e547319a48b491d977a457 [term 1 LEADER]: Becoming Leader. State: Replica: 49898b73a5e547319a48b491d977a457, State: Running, Role: LEADER
I20260811 12:03:05.328755 17212 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49898b73a5e547319a48b491d977a457 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49898b73a5e547319a48b491d977a457" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 33393 } }
I20260811 12:03:05.329396 17202 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49898b73a5e547319a48b491d977a457 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:03:05.331662 17215 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49898b73a5e547319a48b491d977a457 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49898b73a5e547319a48b491d977a457. Latest consensus state: current_term: 1 leader_uuid: "49898b73a5e547319a48b491d977a457" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49898b73a5e547319a48b491d977a457" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 33393 } } }
I20260811 12:03:05.332015 17215 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49898b73a5e547319a48b491d977a457 [sys.catalog]: This master's current role is: LEADER
I20260811 12:03:05.331743 17214 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49898b73a5e547319a48b491d977a457 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "49898b73a5e547319a48b491d977a457" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49898b73a5e547319a48b491d977a457" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 33393 } } }
I20260811 12:03:05.332137 17214 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49898b73a5e547319a48b491d977a457 [sys.catalog]: This master's current role is: LEADER
I20260811 12:03:05.333019 17221 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:03:05.339079 17221 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:03:05.349761 17221 catalog_manager.cc:1347] Generated new cluster ID: 8ef5497ac5494217aa2e5d755fdf8e2d
I20260811 12:03:05.349867 17221 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:03:05.367727 17221 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:03:05.368992 17221 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:03:05.375170 17221 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 49898b73a5e547319a48b491d977a457: Generated new TSK 0
I20260811 12:03:05.376205 17221 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.221.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab.
Entry for principal kudu/127.9.221.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab.
I20260811 12:03:05.433101 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.129:0
--local_ip_for_outbound_sockets=127.9.221.129
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:33393
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab
--principal=kudu/127.9.221.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:37893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:03:05.769519 17243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:03:05.769821 17243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:03:05.769997 17243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:03:05.777311 17243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:03:05.777447 17243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:03:05.777621 17243 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.129
I20260811 12:03:05.786875 17243 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:37893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab
--principal=kudu/127.9.221.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:33393
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.129
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:03:05.788298 17243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:03:05.789599 17243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:03:05.802026 17251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:05.802069 17252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:05.802199 17256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:03:05.803282 17243 server_base.cc:1034] running on GCE node
I20260811 12:03:05.803941 17243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:03:05.804783 17243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:03:05.805984 17243 hybrid_clock.cc:648] HybridClock initialized: now 1786449785805921 us; error 54 us; skew 500 ppm
I20260811 12:03:05.806289 17243 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:03:05 dist-test-slave-mjjr krb5kdc[17069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449785, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:03:05.809393 17243 init.cc:377] Logged in from keytab as kudu/127.9.221.129@KRBTEST.COM (short username kudu)
I20260811 12:03:05.810503 17243 fs_manager.cc:359] Metadata directory not provided
I20260811 12:03:05.810595 17243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:03:05.810847 17243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:03:05.813902 17243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-0/data/instance:
uuid: "fb4953badd1d4cbc96008619e808de31"
format_stamp: "Formatted at 2026-08-11 12:03:05 on dist-test-slave-mjjr"
server_key: "722383a9465af121ad3301b09daafdbf"
server_key_iv: "dfc52c4cee17b8e3917bca9d4b428bbe"
server_key_version: "encryptionkey@0"
I20260811 12:03:05.814635 17243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance:
uuid: "fb4953badd1d4cbc96008619e808de31"
format_stamp: "Formatted at 2026-08-11 12:03:05 on dist-test-slave-mjjr"
server_key: "722383a9465af121ad3301b09daafdbf"
server_key_iv: "dfc52c4cee17b8e3917bca9d4b428bbe"
server_key_version: "encryptionkey@0"
I20260811 12:03:05.819329 17243 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 12:03:05.822754 17264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:05.824060 17243 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:03:05.824226 17243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-0/wal
uuid: "fb4953badd1d4cbc96008619e808de31"
format_stamp: "Formatted at 2026-08-11 12:03:05 on dist-test-slave-mjjr"
server_key: "722383a9465af121ad3301b09daafdbf"
server_key_iv: "dfc52c4cee17b8e3917bca9d4b428bbe"
server_key_version: "encryptionkey@0"
I20260811 12:03:05.824438 17243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:03:05.843070 17243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:03:05.846856 17243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:03:05.847142 17243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:03:05.848242 17243 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:03:05.849581 17243 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:03:05.849664 17243 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:05.849750 17243 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:03:05.849831 17243 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:05.886287 17243 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.129:33251
I20260811 12:03:05.886379 17441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.129:33251 every 8 connection(s)
I20260811 12:03:05.887284 17243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
I20260811 12:03:05.895785 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 17243
I20260811 12:03:05.896016 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance
I20260811 12:03:05.896652 10102 external_mini_cluster.cc:1428] Setting key 5809a9836c70db0b87192b9ab780d795
Aug 11 12:03:05 dist-test-slave-mjjr krb5kdc[17069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449785, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:03:05.907559 17444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:05.889886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:47925 (local address 127.9.221.190:33393)
0811 12:03:05.890298 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:05.890303 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:05.891294 (+ 991us) server_negotiation.cc:407] Connection header received
0811 12:03:05.893090 (+ 1796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:05.893094 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:05.893170 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:05.893264 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:05.894674 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:05.895507 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:05.897241 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:05.897658 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:05.897825 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:05.901745 (+ 3920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:05.901750 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:05.901754 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:05.901786 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:05.904451 (+ 2665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:05.905198 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:05.905203 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:05.905206 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:05.905325 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:05.905780 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:05.905784 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:05.905786 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:05.906058 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:05.906222 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:05.906970 (+ 748us) server_negotiation.cc:299] Negotiation successful
0811 12:03:05.907202 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":207,"threads_started":1}
I20260811 12:03:05.908557 17446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:05.890442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:33393 (local address 127.9.221.129:47925)
0811 12:03:05.891126 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:05.891164 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:05.892846 (+ 1682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:05.893462 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:05.893470 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:05.893960 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:05.894513 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:05.894528 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:05.895727 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:05.895732 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:05.897082 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:05.897094 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:05.897897 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:05.897902 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:05.898082 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:05.898852 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:05.898881 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:05.901523 (+ 2642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:05.904612 (+ 3089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:05.904620 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:05.904635 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:05.905062 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:05.905439 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:05.905444 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:05.905448 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:05.905651 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:05.906198 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:05.906206 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:05.906573 (+ 367us) client_negotiation.cc:769] Sending connection context
0811 12:03:05.906839 (+ 266us) client_negotiation.cc:240] Negotiation successful
0811 12:03:05.907385 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":280,"threads_started":1}
I20260811 12:03:05.910056 17442 heartbeater.cc:347] Connected to a master server at 127.9.221.190:33393
I20260811 12:03:05.910385 17442 heartbeater.cc:464] Registering TS with master...
I20260811 12:03:05.911027 17442 heartbeater.cc:511] Master 127.9.221.190:33393 requested a full tablet report, sending...
I20260811 12:03:05.913062 17145 ts_manager.cc:194] Registered new tserver with Master: fb4953badd1d4cbc96008619e808de31 (127.9.221.129:33251)
I20260811 12:03:05.914469 17145 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.129@KRBTEST.COM'} at 127.9.221.129:47925
WARNING: no policy specified for kudu/127.9.221.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab.
Entry for principal kudu/127.9.221.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab.
I20260811 12:03:05.944656 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.130:0
--local_ip_for_outbound_sockets=127.9.221.130
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:33393
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab
--principal=kudu/127.9.221.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:37893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:03:06.265002 17450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:03:06.265254 17450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:03:06.265348 17450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:03:06.272269 17450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:03:06.272365 17450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:03:06.272481 17450 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.130
I20260811 12:03:06.281678 17450 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:37893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab
--principal=kudu/127.9.221.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:33393
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.130
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:03:06.282912 17450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:03:06.284281 17450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:03:06.297519 17459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:06.297784 17461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:06.297922 17458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:03:06.298302 17450 server_base.cc:1034] running on GCE node
I20260811 12:03:06.298921 17450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:03:06.299893 17450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:03:06.301115 17450 hybrid_clock.cc:648] HybridClock initialized: now 1786449786301059 us; error 48 us; skew 500 ppm
I20260811 12:03:06.301419 17450 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:03:06 dist-test-slave-mjjr krb5kdc[17069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449786, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:03:06.304507 17450 init.cc:377] Logged in from keytab as kudu/127.9.221.130@KRBTEST.COM (short username kudu)
I20260811 12:03:06.305716 17450 fs_manager.cc:359] Metadata directory not provided
I20260811 12:03:06.305804 17450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:03:06.306160 17450 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:03:06.309139 17450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-1/data/instance:
uuid: "05eed86b913a444bbde7729ac74f311d"
format_stamp: "Formatted at 2026-08-11 12:03:06 on dist-test-slave-mjjr"
server_key: "74811984d3597005fbe83e65095aeff4"
server_key_iv: "7dbae2759f781350e09de3c2a5a70ba7"
server_key_version: "encryptionkey@0"
I20260811 12:03:06.309947 17450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance:
uuid: "05eed86b913a444bbde7729ac74f311d"
format_stamp: "Formatted at 2026-08-11 12:03:06 on dist-test-slave-mjjr"
server_key: "74811984d3597005fbe83e65095aeff4"
server_key_iv: "7dbae2759f781350e09de3c2a5a70ba7"
server_key_version: "encryptionkey@0"
I20260811 12:03:06.314852 17450 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.006s
I20260811 12:03:06.318244 17472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:06.319538 17450 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:03:06.319730 17450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-1/wal
uuid: "05eed86b913a444bbde7729ac74f311d"
format_stamp: "Formatted at 2026-08-11 12:03:06 on dist-test-slave-mjjr"
server_key: "74811984d3597005fbe83e65095aeff4"
server_key_iv: "7dbae2759f781350e09de3c2a5a70ba7"
server_key_version: "encryptionkey@0"
I20260811 12:03:06.319949 17450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:03:06.337990 17450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:03:06.342190 17450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:03:06.342463 17450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:03:06.343360 17450 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:03:06.344632 17450 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:03:06.344753 17450 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:06.344882 17450 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:03:06.344972 17450 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:06.378018 17450 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.130:39289
I20260811 12:03:06.378122 17657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.130:39289 every 8 connection(s)
I20260811 12:03:06.378858 17450 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
I20260811 12:03:06.384898 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 17450
I20260811 12:03:06.385090 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance
I20260811 12:03:06.385566 10102 external_mini_cluster.cc:1428] Setting key 5eab33aef9735a2fd1c2144f2370c5de
Aug 11 12:03:06 dist-test-slave-mjjr krb5kdc[17069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449786, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.130@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:03:06.401324 17444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:06.381310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:54753 (local address 127.9.221.190:33393)
0811 12:03:06.381498 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:06.381502 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:06.382568 (+ 1066us) server_negotiation.cc:407] Connection header received
0811 12:03:06.384422 (+ 1854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:06.384429 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:06.384539 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:06.384693 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:06.386457 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:06.387279 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:06.389083 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:06.389867 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:06.390034 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:06.394160 (+ 4126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:06.394169 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:06.394176 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:06.394222 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:06.397980 (+ 3758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:06.398956 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:06.398959 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:06.398962 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:06.399084 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:06.399554 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:06.399557 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:06.399559 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:06.399876 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:06.399999 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:06.400885 (+ 886us) server_negotiation.cc:299] Negotiation successful
0811 12:03:06.401067 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 12:03:06.402901 17661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:06.381742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:33393 (local address 127.9.221.130:54753)
0811 12:03:06.382406 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:06.382443 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:06.384138 (+ 1695us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:06.384923 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:06.384931 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:06.385559 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:06.386257 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:06.386283 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:06.387448 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:06.387453 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:06.388935 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:06.388947 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:06.390092 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:06.390096 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:06.390331 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:06.391228 (+ 897us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:06.391260 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:06.393921 (+ 2661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:06.398126 (+ 4205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:06.398137 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:06.398155 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:06.398826 (+ 671us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:06.399200 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:06.399208 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:06.399214 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:06.399444 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:06.400016 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:06.400026 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:06.400484 (+ 458us) client_negotiation.cc:769] Sending connection context
0811 12:03:06.400809 (+ 325us) client_negotiation.cc:240] Negotiation successful
0811 12:03:06.401407 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":274,"threads_started":1}
I20260811 12:03:06.404428 17658 heartbeater.cc:347] Connected to a master server at 127.9.221.190:33393
I20260811 12:03:06.404752 17658 heartbeater.cc:464] Registering TS with master...
I20260811 12:03:06.405412 17658 heartbeater.cc:511] Master 127.9.221.190:33393 requested a full tablet report, sending...
I20260811 12:03:06.406888 17145 ts_manager.cc:194] Registered new tserver with Master: 05eed86b913a444bbde7729ac74f311d (127.9.221.130:39289)
I20260811 12:03:06.407747 17145 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.130@KRBTEST.COM'} at 127.9.221.130:54753
WARNING: no policy specified for kudu/127.9.221.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab.
Entry for principal kudu/127.9.221.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab.
I20260811 12:03:06.437094 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.131:0
--local_ip_for_outbound_sockets=127.9.221.131
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:33393
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab
--principal=kudu/127.9.221.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:37893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:03:06.756786 17664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:03:06.757102 17664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:03:06.757283 17664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:03:06.764402 17664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:03:06.764556 17664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:03:06.764750 17664 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.131
I20260811 12:03:06.774230 17664 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:37893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab
--principal=kudu/127.9.221.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:33393
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.131
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:03:06.775581 17664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:03:06.776909 17664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:03:06.789570 17670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:06.789535 17669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:06.790134 17673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:03:06.790669 17664 server_base.cc:1034] running on GCE node
I20260811 12:03:06.791332 17664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:03:06.792337 17664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:03:06.793560 17664 hybrid_clock.cc:648] HybridClock initialized: now 1786449786793496 us; error 60 us; skew 500 ppm
I20260811 12:03:06.793884 17664 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:03:06 dist-test-slave-mjjr krb5kdc[17069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449786, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:03:06.796983 17664 init.cc:377] Logged in from keytab as kudu/127.9.221.131@KRBTEST.COM (short username kudu)
I20260811 12:03:06.798087 17664 fs_manager.cc:359] Metadata directory not provided
I20260811 12:03:06.798240 17664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:03:06.798560 17664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:03:06.801451 17664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-2/data/instance:
uuid: "1d2780ae4efa495d9562e5857e5a896a"
format_stamp: "Formatted at 2026-08-11 12:03:06 on dist-test-slave-mjjr"
server_key: "5f5a19503d5ffecab1fac895db33c005"
server_key_iv: "435a0d93cdcf06e4397e7c11898bdd1c"
server_key_version: "encryptionkey@0"
I20260811 12:03:06.802244 17664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance:
uuid: "1d2780ae4efa495d9562e5857e5a896a"
format_stamp: "Formatted at 2026-08-11 12:03:06 on dist-test-slave-mjjr"
server_key: "5f5a19503d5ffecab1fac895db33c005"
server_key_iv: "435a0d93cdcf06e4397e7c11898bdd1c"
server_key_version: "encryptionkey@0"
I20260811 12:03:06.807155 17664 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 12:03:06.810423 17683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:06.811600 17664 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 12:03:06.811820 17664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-2/wal
uuid: "1d2780ae4efa495d9562e5857e5a896a"
format_stamp: "Formatted at 2026-08-11 12:03:06 on dist-test-slave-mjjr"
server_key: "5f5a19503d5ffecab1fac895db33c005"
server_key_iv: "435a0d93cdcf06e4397e7c11898bdd1c"
server_key_version: "encryptionkey@0"
I20260811 12:03:06.812090 17664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:03:06.830904 17664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:03:06.834636 17664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:03:06.834896 17664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:03:06.835850 17664 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:03:06.837086 17664 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:03:06.837224 17664 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:06.837345 17664 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:03:06.837438 17664 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:06.870293 17664 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.131:42885
I20260811 12:03:06.870379 17869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.131:42885 every 8 connection(s)
I20260811 12:03:06.871219 17664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
I20260811 12:03:06.875254 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 17664
I20260811 12:03:06.875447 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance
I20260811 12:03:06.876063 10102 external_mini_cluster.cc:1428] Setting key 7570337a1775d4e09bd0e2bff119ea2f
Aug 11 12:03:06 dist-test-slave-mjjr krb5kdc[17069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449786, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.131@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:03:06.892355 17444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:06.873881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:58019 (local address 127.9.221.190:33393)
0811 12:03:06.874151 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:06.874159 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:06.875566 (+ 1407us) server_negotiation.cc:407] Connection header received
0811 12:03:06.877902 (+ 2336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:06.877905 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:06.878014 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:06.878120 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:06.879922 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:06.880748 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:06.882179 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:06.882594 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:06.882683 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:06.886255 (+ 3572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:06.886259 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:06.886262 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:06.886292 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:06.888871 (+ 2579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:06.889772 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:06.889780 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:06.889787 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:06.889950 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:06.890442 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:06.890447 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:06.890458 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:06.890814 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:06.891279 (+ 465us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:06.891780 (+ 501us) server_negotiation.cc:299] Negotiation successful
0811 12:03:06.892013 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 12:03:06.893044 17872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:06.874376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:33393 (local address 127.9.221.131:58019)
0811 12:03:06.875364 (+ 988us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:06.875428 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:06.877584 (+ 2156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:06.878496 (+ 912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:06.878504 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:06.879018 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:06.879760 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:06.879779 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:06.880864 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:06.880868 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:06.882057 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:06.882070 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:06.882773 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:06.882777 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:06.882958 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:06.883680 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:06.883713 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:06.886082 (+ 2369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:06.889010 (+ 2928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:06.889018 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:06.889032 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:06.889588 (+ 556us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:06.890067 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:06.890075 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:06.890078 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:06.890315 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:06.890966 (+ 651us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:06.890974 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:06.891319 (+ 345us) client_negotiation.cc:769] Sending connection context
0811 12:03:06.891622 (+ 303us) client_negotiation.cc:240] Negotiation successful
0811 12:03:06.892028 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"thread_start_us":388,"threads_started":1}
I20260811 12:03:06.894373 17870 heartbeater.cc:347] Connected to a master server at 127.9.221.190:33393
I20260811 12:03:06.894632 17870 heartbeater.cc:464] Registering TS with master...
I20260811 12:03:06.895201 17870 heartbeater.cc:511] Master 127.9.221.190:33393 requested a full tablet report, sending...
I20260811 12:03:06.896620 17145 ts_manager.cc:194] Registered new tserver with Master: 1d2780ae4efa495d9562e5857e5a896a (127.9.221.131:42885)
I20260811 12:03:06.897403 17145 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.131@KRBTEST.COM'} at 127.9.221.131:58019
I20260811 12:03:06.903350 10102 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:03:06.917236 17442 heartbeater.cc:503] Master 127.9.221.190:33393 was elected leader, sending a full tablet report...
I20260811 12:03:06.920742 17444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:06.907957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56388 (local address 127.9.221.190:33393)
0811 12:03:06.908111 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:06.908115 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:06.908660 (+ 545us) server_negotiation.cc:407] Connection header received
0811 12:03:06.908847 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:06.908852 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:06.908925 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:06.909026 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:06.910435 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:06.911126 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:06.912402 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:06.912851 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:06.912998 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:06.915350 (+ 2352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:06.915354 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:06.915358 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:06.915388 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:06.918197 (+ 2809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:06.918920 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:06.918926 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:06.918929 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:06.919101 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:06.919417 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:06.919420 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:06.919422 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:06.919777 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:06.919965 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:06.920236 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 12:03:06.920392 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 12:03:06.924548 17145 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56388:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:03:06.931722 17145 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:03:06.960122 17888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:06.944046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:33251 (local address 127.0.0.1:60490)
0811 12:03:06.945053 (+ 1007us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:06.945096 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:06.945289 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:06.946306 (+ 1017us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:06.946311 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:06.946332 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:03:06.948290 (+ 1958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:06.948320 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:06.952287 (+ 3967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:06.952296 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:06.957012 (+ 4716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:06.957035 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:06.959007 (+ 1972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:06.959013 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:06.959223 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:06.959279 (+ 56us) client_negotiation.cc:769] Sending connection context
0811 12:03:06.959421 (+ 142us) client_negotiation.cc:240] Negotiation successful
0811 12:03:06.959691 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":778,"thread_start_us":645,"threads_started":1}
I20260811 12:03:06.960665 17889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:06.944243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60490 (local address 127.9.221.129:33251)
0811 12:03:06.945124 (+ 881us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:06.945131 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:06.945180 (+ 49us) server_negotiation.cc:407] Connection header received
0811 12:03:06.945546 (+ 366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:06.945555 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:06.945820 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:06.946035 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:03:06.948567 (+ 2532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:06.952086 (+ 3519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:06.957270 (+ 5184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:06.958790 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:06.958983 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:06.959238 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:06.960115 (+ 877us) server_negotiation.cc:299] Negotiation successful
0811 12:03:06.960308 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":726,"thread_start_us":566,"threads_started":1}
I20260811 12:03:06.964808 17344 tablet_service.cc:1467] Processing CreateTablet for tablet 5bf84c45ce234638bffe6d1fb03353f3 (DEFAULT_TABLE table=test-table [id=39853a5b6ee2447ab7a97c16917fa4ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:03:06.966413 17893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:06.946692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35708 (local address 127.9.221.130:39289)
0811 12:03:06.958446 (+ 11754us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:06.958458 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:06.958497 (+ 39us) server_negotiation.cc:407] Connection header received
0811 12:03:06.958632 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:06.958645 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:06.958985 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:06.959368 (+ 383us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:03:06.960003 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:06.962120 (+ 2117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:06.964285 (+ 2165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:06.965187 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:06.965363 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:06.965567 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:06.965919 (+ 352us) server_negotiation.cc:299] Negotiation successful
0811 12:03:06.966078 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11563,"thread_start_us":11431,"threads_started":1}
I20260811 12:03:06.966408 17892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:06.946084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.130:39289 (local address 127.0.0.1:35708)
0811 12:03:06.958126 (+ 12042us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:06.958150 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:06.958277 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:06.959332 (+ 1055us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:06.959336 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:06.959364 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:03:06.959790 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:06.959803 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:06.962266 (+ 2463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:06.962272 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:06.964117 (+ 1845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:06.964130 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:06.965422 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:06.965428 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:06.965688 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:06.965739 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 12:03:06.965849 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 12:03:06.965984 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11882,"mutex_wait_us":40,"thread_start_us":11747,"threads_started":1}
I20260811 12:03:06.966439 17344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bf84c45ce234638bffe6d1fb03353f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:03:06.969475 17890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:06.945513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.131:42885 (local address 127.0.0.1:44418)
0811 12:03:06.946036 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:06.946053 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:06.946156 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:06.960729 (+ 14573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:06.960736 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:06.960763 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:03:06.961179 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:06.961193 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:06.963598 (+ 2405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:06.963603 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:06.965748 (+ 2145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:06.965763 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:06.968641 (+ 2878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:06.968648 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:06.968824 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:06.968861 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 12:03:06.968963 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 12:03:06.969096 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"mutex_wait_us":24,"thread_start_us":271,"threads_started":1}
I20260811 12:03:06.969563 17891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:06.945912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44418 (local address 127.9.221.131:42885)
0811 12:03:06.960124 (+ 14212us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:06.960132 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:06.960162 (+ 30us) server_negotiation.cc:407] Connection header received
0811 12:03:06.960249 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:06.960267 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:06.960494 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:06.961411 (+ 917us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:03:06.961731 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:06.963433 (+ 1702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:06.965960 (+ 2527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:06.968422 (+ 2462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:06.968569 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:06.968796 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:06.969033 (+ 237us) server_negotiation.cc:299] Negotiation successful
0811 12:03:06.969204 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14050,"thread_start_us":13901,"threads_started":1}
I20260811 12:03:06.970158 17551 tablet_service.cc:1467] Processing CreateTablet for tablet 5bf84c45ce234638bffe6d1fb03353f3 (DEFAULT_TABLE table=test-table [id=39853a5b6ee2447ab7a97c16917fa4ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:03:06.971556 17551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bf84c45ce234638bffe6d1fb03353f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:03:06.973313 17764 tablet_service.cc:1467] Processing CreateTablet for tablet 5bf84c45ce234638bffe6d1fb03353f3 (DEFAULT_TABLE table=test-table [id=39853a5b6ee2447ab7a97c16917fa4ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:03:06.974746 17764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bf84c45ce234638bffe6d1fb03353f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:03:06.978698 17894 tablet_bootstrap.cc:492] T 5bf84c45ce234638bffe6d1fb03353f3 P fb4953badd1d4cbc96008619e808de31: Bootstrap starting.
I20260811 12:03:06.981669 17894 tablet_bootstrap.cc:654] T 5bf84c45ce234638bffe6d1fb03353f3 P fb4953badd1d4cbc96008619e808de31: Neither blocks nor log segments found. Creating new log.
I20260811 12:03:06.983305 17894 log.cc:824] T 5bf84c45ce234638bffe6d1fb03353f3 P fb4953badd1d4cbc96008619e808de31: Log is configured to *not* fsync() on all Append() calls
I20260811 12:03:06.986025 17897 tablet_bootstrap.cc:492] T 5bf84c45ce234638bffe6d1fb03353f3 P 05eed86b913a444bbde7729ac74f311d: Bootstrap starting.
I20260811 12:03:06.986522 17894 tablet_bootstrap.cc:492] T 5bf84c45ce234638bffe6d1fb03353f3 P fb4953badd1d4cbc96008619e808de31: No bootstrap required, opened a new log
I20260811 12:03:06.986877 17894 ts_tablet_manager.cc:1397] T 5bf84c45ce234638bffe6d1fb03353f3 P fb4953badd1d4cbc96008619e808de31: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260811 12:03:06.989050 17897 tablet_bootstrap.cc:654] T 5bf84c45ce234638bffe6d1fb03353f3 P 05eed86b913a444bbde7729ac74f311d: Neither blocks nor log segments found. Creating new log.
I20260811 12:03:06.990253 17897 log.cc:824] T 5bf84c45ce234638bffe6d1fb03353f3 P 05eed86b913a444bbde7729ac74f311d: Log is configured to *not* fsync() on all Append() calls
I20260811 12:03:06.990715 17898 tablet_bootstrap.cc:492] T 5bf84c45ce234638bffe6d1fb03353f3 P 1d2780ae4efa495d9562e5857e5a896a: Bootstrap starting.
I20260811 12:03:06.994397 17898 tablet_bootstrap.cc:654] T 5bf84c45ce234638bffe6d1fb03353f3 P 1d2780ae4efa495d9562e5857e5a896a: Neither blocks nor log segments found. Creating new log.
I20260811 12:03:06.994642 17897 tablet_bootstrap.cc:492] T 5bf84c45ce234638bffe6d1fb03353f3 P 05eed86b913a444bbde7729ac74f311d: No bootstrap required, opened a new log
I20260811 12:03:06.994938 17897 ts_tablet_manager.cc:1397] T 5bf84c45ce234638bffe6d1fb03353f3 P 05eed86b913a444bbde7729ac74f311d: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260811 12:03:06.995707 17898 log.cc:824] T 5bf84c45ce234638bffe6d1fb03353f3 P 1d2780ae4efa495d9562e5857e5a896a: Log is configured to *not* fsync() on all Append() calls
I20260811 12:03:06.997215 17894 raft_consensus.cc:348] T 5bf84c45ce234638bffe6d1fb03353f3 P fb4953badd1d4cbc96008619e808de31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb4953badd1d4cbc96008619e808de31" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33251 } } peers { permanent_uuid: "1d2780ae4efa495d9562e5857e5a896a" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 42885 } } peers { permanent_uuid: "05eed86b913a444bbde7729ac74f311d" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 39289 } }
I20260811 12:03:06.997583 17894 raft_consensus.cc:374] T 5bf84c45ce234638bffe6d1fb03353f3 P fb4953badd1d4cbc96008619e808de31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:03:06.997707 17894 raft_consensus.cc:729] T 5bf84c45ce234638bffe6d1fb03353f3 P fb4953badd1d4cbc96008619e808de31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb4953badd1d4cbc96008619e808de31, State: Initialized, Role: FOLLOWER
I20260811 12:03:06.998319 17894 consensus_queue.cc:260] T 5bf84c45ce234638bffe6d1fb03353f3 P fb4953badd1d4cbc96008619e808de31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb4953badd1d4cbc96008619e808de31" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33251 } } peers { permanent_uuid: "1d2780ae4efa495d9562e5857e5a896a" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 42885 } } peers { permanent_uuid: "05eed86b913a444bbde7729ac74f311d" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 39289 } }
I20260811 12:03:06.998793 17898 tablet_bootstrap.cc:492] T 5bf84c45ce234638bffe6d1fb03353f3 P 1d2780ae4efa495d9562e5857e5a896a: No bootstrap required, opened a new log
I20260811 12:03:06.999115 17898 ts_tablet_manager.cc:1397] T 5bf84c45ce234638bffe6d1fb03353f3 P 1d2780ae4efa495d9562e5857e5a896a: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260811 12:03:07.000000 17894 ts_tablet_manager.cc:1428] T 5bf84c45ce234638bffe6d1fb03353f3 P fb4953badd1d4cbc96008619e808de31: Time spent starting tablet: real 0.013s user 0.011s sys 0.001s
I20260811 12:03:07.006132 17897 raft_consensus.cc:348] T 5bf84c45ce234638bffe6d1fb03353f3 P 05eed86b913a444bbde7729ac74f311d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb4953badd1d4cbc96008619e808de31" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33251 } } peers { permanent_uuid: "1d2780ae4efa495d9562e5857e5a896a" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 42885 } } peers { permanent_uuid: "05eed86b913a444bbde7729ac74f311d" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 39289 } }
I20260811 12:03:07.006629 17897 raft_consensus.cc:374] T 5bf84c45ce234638bffe6d1fb03353f3 P 05eed86b913a444bbde7729ac74f311d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:03:07.006747 17897 raft_consensus.cc:729] T 5bf84c45ce234638bffe6d1fb03353f3 P 05eed86b913a444bbde7729ac74f311d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05eed86b913a444bbde7729ac74f311d, State: Initialized, Role: FOLLOWER
I20260811 12:03:07.007279 17897 consensus_queue.cc:260] T 5bf84c45ce234638bffe6d1fb03353f3 P 05eed86b913a444bbde7729ac74f311d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb4953badd1d4cbc96008619e808de31" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33251 } } peers { permanent_uuid: "1d2780ae4efa495d9562e5857e5a896a" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 42885 } } peers { permanent_uuid: "05eed86b913a444bbde7729ac74f311d" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 39289 } }
I20260811 12:03:07.007805 17898 raft_consensus.cc:348] T 5bf84c45ce234638bffe6d1fb03353f3 P 1d2780ae4efa495d9562e5857e5a896a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb4953badd1d4cbc96008619e808de31" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33251 } } peers { permanent_uuid: "1d2780ae4efa495d9562e5857e5a896a" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 42885 } } peers { permanent_uuid: "05eed86b913a444bbde7729ac74f311d" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 39289 } }
I20260811 12:03:07.008226 17898 raft_consensus.cc:374] T 5bf84c45ce234638bffe6d1fb03353f3 P 1d2780ae4efa495d9562e5857e5a896a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:03:07.008373 17898 raft_consensus.cc:729] T 5bf84c45ce234638bffe6d1fb03353f3 P 1d2780ae4efa495d9562e5857e5a896a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d2780ae4efa495d9562e5857e5a896a, State: Initialized, Role: FOLLOWER
I20260811 12:03:07.008453 17658 heartbeater.cc:503] Master 127.9.221.190:33393 was elected leader, sending a full tablet report...
I20260811 12:03:07.008931 17898 consensus_queue.cc:260] T 5bf84c45ce234638bffe6d1fb03353f3 P 1d2780ae4efa495d9562e5857e5a896a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb4953badd1d4cbc96008619e808de31" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33251 } } peers { permanent_uuid: "1d2780ae4efa495d9562e5857e5a896a" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 42885 } } peers { permanent_uuid: "05eed86b913a444bbde7729ac74f311d" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 39289 } }
I20260811 12:03:07.009066 17897 ts_tablet_manager.cc:1428] T 5bf84c45ce234638bffe6d1fb03353f3 P 05eed86b913a444bbde7729ac74f311d: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260811 12:03:07.010002 17870 heartbeater.cc:503] Master 127.9.221.190:33393 was elected leader, sending a full tablet report...
I20260811 12:03:07.010601 17898 ts_tablet_manager.cc:1428] T 5bf84c45ce234638bffe6d1fb03353f3 P 1d2780ae4efa495d9562e5857e5a896a: Time spent starting tablet: real 0.011s user 0.006s sys 0.004s
I20260811 12:03:07.029563 17904 raft_consensus.cc:482] T 5bf84c45ce234638bffe6d1fb03353f3 P fb4953badd1d4cbc96008619e808de31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:03:07.029852 17904 raft_consensus.cc:504] T 5bf84c45ce234638bffe6d1fb03353f3 P fb4953badd1d4cbc96008619e808de31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb4953badd1d4cbc96008619e808de31" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33251 } } peers { permanent_uuid: "1d2780ae4efa495d9562e5857e5a896a" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 42885 } } peers { permanent_uuid: "05eed86b913a444bbde7729ac74f311d" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 39289 } }
I20260811 12:03:07.031497 17904 leader_election.cc:288] T 5bf84c45ce234638bffe6d1fb03353f3 P fb4953badd1d4cbc96008619e808de31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d2780ae4efa495d9562e5857e5a896a (127.9.221.131:42885), 05eed86b913a444bbde7729ac74f311d (127.9.221.130:39289)
I20260811 12:03:07.037891 17891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:07.031761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:36359 (local address 127.9.221.131:42885)
0811 12:03:07.031934 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:07.031938 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:07.032544 (+ 606us) server_negotiation.cc:407] Connection header received
0811 12:03:07.032771 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:07.032781 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:07.032880 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:07.032972 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:03:07.033555 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:07.034269 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:07.036107 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:07.036933 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:07.037105 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:07.037182 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:07.037497 (+ 315us) server_negotiation.cc:299] Negotiation successful
0811 12:03:07.037623 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 12:03:07.038072 17907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:07.031843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.131:42885 (local address 127.9.221.129:36359)
0811 12:03:07.032404 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:07.032419 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:07.032593 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:07.033051 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:07.033055 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:07.033084 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:03:07.033416 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:07.033425 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:07.034396 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:07.034403 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:07.035959 (+ 1556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:07.035970 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:07.037138 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:07.037144 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:07.037297 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:07.037355 (+ 58us) client_negotiation.cc:769] Sending connection context
0811 12:03:07.037473 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 12:03:07.037605 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"mutex_wait_us":42,"thread_start_us":307,"threads_started":1}
I20260811 12:03:07.038370 17893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:07.032782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:53257 (local address 127.9.221.130:39289)
0811 12:03:07.032979 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:07.032986 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:07.033242 (+ 256us) server_negotiation.cc:407] Connection header received
0811 12:03:07.033449 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:07.033456 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:07.033514 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:07.033606 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:03:07.034121 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:07.034819 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:07.036547 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:07.037385 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:07.037576 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:07.037647 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:07.038009 (+ 362us) server_negotiation.cc:299] Negotiation successful
0811 12:03:07.038126 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 12:03:07.039085 17908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:07.032540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.130:39289 (local address 127.9.221.129:53257)
0811 12:03:07.033142 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:07.033158 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:07.033309 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:07.033619 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:07.033623 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:07.033646 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:03:07.033978 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:07.033986 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:07.034964 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:07.034968 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:07.036392 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:07.036404 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:07.037607 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:07.037613 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:07.037779 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:07.037802 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 12:03:07.038559 (+ 757us) client_negotiation.cc:240] Negotiation successful
0811 12:03:07.038697 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"mutex_wait_us":3,"thread_start_us":299,"threads_started":1}
I20260811 12:03:07.039170 17801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bf84c45ce234638bffe6d1fb03353f3" candidate_uuid: "fb4953badd1d4cbc96008619e808de31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d2780ae4efa495d9562e5857e5a896a" is_pre_election: true
I20260811 12:03:07.039762 17801 raft_consensus.cc:2454] T 5bf84c45ce234638bffe6d1fb03353f3 P 1d2780ae4efa495d9562e5857e5a896a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb4953badd1d4cbc96008619e808de31 in term 0.
I20260811 12:03:07.040117 17592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bf84c45ce234638bffe6d1fb03353f3" candidate_uuid: "fb4953badd1d4cbc96008619e808de31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05eed86b913a444bbde7729ac74f311d" is_pre_election: true
I20260811 12:03:07.040462 17268 leader_election.cc:302] T 5bf84c45ce234638bffe6d1fb03353f3 P fb4953badd1d4cbc96008619e808de31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d2780ae4efa495d9562e5857e5a896a, fb4953badd1d4cbc96008619e808de31; no voters:
I20260811 12:03:07.040570 17592 raft_consensus.cc:2454] T 5bf84c45ce234638bffe6d1fb03353f3 P 05eed86b913a444bbde7729ac74f311d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb4953badd1d4cbc96008619e808de31 in term 0.
I20260811 12:03:07.040823 17904 raft_consensus.cc:2781] T 5bf84c45ce234638bffe6d1fb03353f3 P fb4953badd1d4cbc96008619e808de31 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:03:07.040964 17904 raft_consensus.cc:482] T 5bf84c45ce234638bffe6d1fb03353f3 P fb4953badd1d4cbc96008619e808de31 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:03:07.041117 17904 raft_consensus.cc:3037] T 5bf84c45ce234638bffe6d1fb03353f3 P fb4953badd1d4cbc96008619e808de31 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:03:07.043478 17904 raft_consensus.cc:504] T 5bf84c45ce234638bffe6d1fb03353f3 P fb4953badd1d4cbc96008619e808de31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb4953badd1d4cbc96008619e808de31" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33251 } } peers { permanent_uuid: "1d2780ae4efa495d9562e5857e5a896a" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 42885 } } peers { permanent_uuid: "05eed86b913a444bbde7729ac74f311d" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 39289 } }
I20260811 12:03:07.044035 17904 leader_election.cc:288] T 5bf84c45ce234638bffe6d1fb03353f3 P fb4953badd1d4cbc96008619e808de31 [CANDIDATE]: Term 1 election: Requested vote from peers 1d2780ae4efa495d9562e5857e5a896a (127.9.221.131:42885), 05eed86b913a444bbde7729ac74f311d (127.9.221.130:39289)
I20260811 12:03:07.044456 17801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bf84c45ce234638bffe6d1fb03353f3" candidate_uuid: "fb4953badd1d4cbc96008619e808de31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d2780ae4efa495d9562e5857e5a896a"
I20260811 12:03:07.044498 17592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bf84c45ce234638bffe6d1fb03353f3" candidate_uuid: "fb4953badd1d4cbc96008619e808de31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05eed86b913a444bbde7729ac74f311d"
I20260811 12:03:07.044625 17801 raft_consensus.cc:3037] T 5bf84c45ce234638bffe6d1fb03353f3 P 1d2780ae4efa495d9562e5857e5a896a [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:03:07.044699 17592 raft_consensus.cc:3037] T 5bf84c45ce234638bffe6d1fb03353f3 P 05eed86b913a444bbde7729ac74f311d [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:03:07.046860 17801 raft_consensus.cc:2454] T 5bf84c45ce234638bffe6d1fb03353f3 P 1d2780ae4efa495d9562e5857e5a896a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb4953badd1d4cbc96008619e808de31 in term 1.
I20260811 12:03:07.047032 17592 raft_consensus.cc:2454] T 5bf84c45ce234638bffe6d1fb03353f3 P 05eed86b913a444bbde7729ac74f311d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb4953badd1d4cbc96008619e808de31 in term 1.
I20260811 12:03:07.047257 17268 leader_election.cc:302] T 5bf84c45ce234638bffe6d1fb03353f3 P fb4953badd1d4cbc96008619e808de31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d2780ae4efa495d9562e5857e5a896a, fb4953badd1d4cbc96008619e808de31; no voters:
I20260811 12:03:07.047535 17904 raft_consensus.cc:2781] T 5bf84c45ce234638bffe6d1fb03353f3 P fb4953badd1d4cbc96008619e808de31 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:03:07.048070 17904 raft_consensus.cc:686] T 5bf84c45ce234638bffe6d1fb03353f3 P fb4953badd1d4cbc96008619e808de31 [term 1 LEADER]: Becoming Leader. State: Replica: fb4953badd1d4cbc96008619e808de31, State: Running, Role: LEADER
I20260811 12:03:07.048605 17904 consensus_queue.cc:237] T 5bf84c45ce234638bffe6d1fb03353f3 P fb4953badd1d4cbc96008619e808de31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb4953badd1d4cbc96008619e808de31" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33251 } } peers { permanent_uuid: "1d2780ae4efa495d9562e5857e5a896a" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 42885 } } peers { permanent_uuid: "05eed86b913a444bbde7729ac74f311d" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 39289 } }
I20260811 12:03:07.052917 17144 catalog_manager.cc:5495] T 5bf84c45ce234638bffe6d1fb03353f3 P fb4953badd1d4cbc96008619e808de31 reported cstate change: term changed from 0 to 1, leader changed from <none> to fb4953badd1d4cbc96008619e808de31 (127.9.221.129). New cstate: current_term: 1 leader_uuid: "fb4953badd1d4cbc96008619e808de31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb4953badd1d4cbc96008619e808de31" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1d2780ae4efa495d9562e5857e5a896a" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 42885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05eed86b913a444bbde7729ac74f311d" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 39289 } health_report { overall_health: UNKNOWN } } }
I20260811 12:03:07.077030 17889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:07.072129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60500 (local address 127.9.221.129:33251)
0811 12:03:07.072372 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:07.072377 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:07.072393 (+ 16us) server_negotiation.cc:407] Connection header received
0811 12:03:07.072588 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:07.072592 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:07.072672 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:07.072790 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:03:07.073108 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:07.073808 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:07.075074 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:07.075475 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:07.075563 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:07.075999 (+ 436us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:03:07.076489 (+ 490us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:03:07.076590 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:07.076671 (+ 81us) server_negotiation.cc:299] Negotiation successful
0811 12:03:07.076732 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 12:03:07.084764 17592 raft_consensus.cc:1261] T 5bf84c45ce234638bffe6d1fb03353f3 P 05eed86b913a444bbde7729ac74f311d [term 1 FOLLOWER]: Refusing update from remote peer fb4953badd1d4cbc96008619e808de31: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:03:07.084764 17801 raft_consensus.cc:1261] T 5bf84c45ce234638bffe6d1fb03353f3 P 1d2780ae4efa495d9562e5857e5a896a [term 1 FOLLOWER]: Refusing update from remote peer fb4953badd1d4cbc96008619e808de31: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:03:07.085726 17909 consensus_queue.cc:1035] T 5bf84c45ce234638bffe6d1fb03353f3 P fb4953badd1d4cbc96008619e808de31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d2780ae4efa495d9562e5857e5a896a" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 42885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:03:07.086097 17904 consensus_queue.cc:1035] T 5bf84c45ce234638bffe6d1fb03353f3 P fb4953badd1d4cbc96008619e808de31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05eed86b913a444bbde7729ac74f311d" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 39289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:03:07.110914 17145 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56388:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:03:07.111229 17145 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56388:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:03:07.114966 17145 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 49898b73a5e547319a48b491d977a457: Sending DeleteTablet for 3 replicas of tablet 5bf84c45ce234638bffe6d1fb03353f3
I20260811 12:03:07.116189 17344 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bf84c45ce234638bffe6d1fb03353f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:03:07 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:60490
I20260811 12:03:07.116376 17551 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bf84c45ce234638bffe6d1fb03353f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:03:07 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:35708
I20260811 12:03:07.116376 17764 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bf84c45ce234638bffe6d1fb03353f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:03:07 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:44418
I20260811 12:03:07.118953 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 17243
I20260811 12:03:07.133594 17934 tablet_replica.cc:330] T 5bf84c45ce234638bffe6d1fb03353f3 P 05eed86b913a444bbde7729ac74f311d: stopping tablet replica
I20260811 12:03:07.133757 17935 tablet_replica.cc:330] T 5bf84c45ce234638bffe6d1fb03353f3 P 1d2780ae4efa495d9562e5857e5a896a: stopping tablet replica
I20260811 12:03:07.134152 17934 raft_consensus.cc:2229] T 5bf84c45ce234638bffe6d1fb03353f3 P 05eed86b913a444bbde7729ac74f311d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:03:07.134203 17935 raft_consensus.cc:2229] T 5bf84c45ce234638bffe6d1fb03353f3 P 1d2780ae4efa495d9562e5857e5a896a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:03:07.134748 17934 raft_consensus.cc:2258] T 5bf84c45ce234638bffe6d1fb03353f3 P 05eed86b913a444bbde7729ac74f311d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:03:07.134739 17935 raft_consensus.cc:2258] T 5bf84c45ce234638bffe6d1fb03353f3 P 1d2780ae4efa495d9562e5857e5a896a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:03:07.135195 17932 tablet_replica.cc:330] T 5bf84c45ce234638bffe6d1fb03353f3 P fb4953badd1d4cbc96008619e808de31: stopping tablet replica
I20260811 12:03:07.135624 17932 raft_consensus.cc:2229] T 5bf84c45ce234638bffe6d1fb03353f3 P fb4953badd1d4cbc96008619e808de31 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:03:07.136469 17932 raft_consensus.cc:2258] T 5bf84c45ce234638bffe6d1fb03353f3 P fb4953badd1d4cbc96008619e808de31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:03:07.136905 17934 ts_tablet_manager.cc:1905] T 5bf84c45ce234638bffe6d1fb03353f3 P 05eed86b913a444bbde7729ac74f311d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:03:07.139137 17935 ts_tablet_manager.cc:1905] T 5bf84c45ce234638bffe6d1fb03353f3 P 1d2780ae4efa495d9562e5857e5a896a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:03:07.140637 17932 ts_tablet_manager.cc:1905] T 5bf84c45ce234638bffe6d1fb03353f3 P fb4953badd1d4cbc96008619e808de31: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:03:07.143716 17934 ts_tablet_manager.cc:1918] T 5bf84c45ce234638bffe6d1fb03353f3 P 05eed86b913a444bbde7729ac74f311d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:03:07.143942 17934 log.cc:1196] T 5bf84c45ce234638bffe6d1fb03353f3 P 05eed86b913a444bbde7729ac74f311d: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-1/wal/wals/5bf84c45ce234638bffe6d1fb03353f3
Aug 11 12:03:07 dist-test-slave-mjjr krb5kdc[17069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449784, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.129@KRBTEST.COM
I20260811 12:03:07.144574 17934 ts_tablet_manager.cc:1939] T 5bf84c45ce234638bffe6d1fb03353f3 P 05eed86b913a444bbde7729ac74f311d: Deleting consensus metadata
I20260811 12:03:07.146490 17117 catalog_manager.cc:4841] TS 05eed86b913a444bbde7729ac74f311d (127.9.221.130:39289): tablet 5bf84c45ce234638bffe6d1fb03353f3 (table test-table [id=39853a5b6ee2447ab7a97c16917fa4ad]) successfully deleted
I20260811 12:03:07.146132 17935 ts_tablet_manager.cc:1918] T 5bf84c45ce234638bffe6d1fb03353f3 P 1d2780ae4efa495d9562e5857e5a896a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:03:07.147032 17935 log.cc:1196] T 5bf84c45ce234638bffe6d1fb03353f3 P 1d2780ae4efa495d9562e5857e5a896a: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-2/wal/wals/5bf84c45ce234638bffe6d1fb03353f3
I20260811 12:03:07.147574 17935 ts_tablet_manager.cc:1939] T 5bf84c45ce234638bffe6d1fb03353f3 P 1d2780ae4efa495d9562e5857e5a896a: Deleting consensus metadata
I20260811 12:03:07.149166 17117 catalog_manager.cc:4841] TS 1d2780ae4efa495d9562e5857e5a896a (127.9.221.131:42885): tablet 5bf84c45ce234638bffe6d1fb03353f3 (table test-table [id=39853a5b6ee2447ab7a97c16917fa4ad]) successfully deleted
I20260811 12:03:07.151454 17932 ts_tablet_manager.cc:1918] T 5bf84c45ce234638bffe6d1fb03353f3 P fb4953badd1d4cbc96008619e808de31: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:03:07.151665 17932 log.cc:1196] T 5bf84c45ce234638bffe6d1fb03353f3 P fb4953badd1d4cbc96008619e808de31: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449743084027-10102-0/minicluster-data/ts-0/wal/wals/5bf84c45ce234638bffe6d1fb03353f3
I20260811 12:03:07.152269 17932 ts_tablet_manager.cc:1939] T 5bf84c45ce234638bffe6d1fb03353f3 P fb4953badd1d4cbc96008619e808de31: Deleting consensus metadata
I20260811 12:03:07.153926 17117 catalog_manager.cc:4841] TS fb4953badd1d4cbc96008619e808de31 (127.9.221.129:33251): tablet 5bf84c45ce234638bffe6d1fb03353f3 (table test-table [id=39853a5b6ee2447ab7a97c16917fa4ad]) successfully deleted
I20260811 12:03:07.158887 17889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:07.121215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60516 (local address 127.9.221.129:33251)
0811 12:03:07.121464 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:07.121473 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:07.125876 (+ 4403us) server_negotiation.cc:407] Connection header received
0811 12:03:07.126717 (+ 841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:07.126724 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:07.126812 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:07.126888 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:07.130364 (+ 3476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:07.131618 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:07.136757 (+ 5139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:07.137530 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:07.137651 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:07.152720 (+ 15069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:07.152733 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:07.152747 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:07.152789 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:07.155925 (+ 3136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:07.156684 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:07.156694 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:07.156696 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:07.156845 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:07.157247 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:07.157251 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:07.157253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:07.157632 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:07.157962 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:07.158265 (+ 303us) server_negotiation.cc:299] Negotiation successful
0811 12:03:07.158531 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 12:03:07.159584 17433 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:03:07.273358 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 17243
I20260811 12:03:07.294782 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 17450
Aug 11 12:03:07 dist-test-slave-mjjr krb5kdc[17069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449784, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.130@KRBTEST.COM
I20260811 12:03:07.310860 17893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:07.295593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35710 (local address 127.9.221.130:39289)
0811 12:03:07.295805 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:07.295811 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:07.295834 (+ 23us) server_negotiation.cc:407] Connection header received
0811 12:03:07.296014 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:07.296020 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:07.296107 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:07.296211 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:07.297409 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:07.298565 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:07.299965 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:07.300558 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:07.300680 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:07.304711 (+ 4031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:07.304729 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:07.304748 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:07.304787 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:07.308415 (+ 3628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:07.309059 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:07.309068 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:07.309072 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:07.309218 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:07.309565 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:07.309571 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:07.309577 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:07.309987 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:07.310202 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:07.310460 (+ 258us) server_negotiation.cc:299] Negotiation successful
0811 12:03:07.310632 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260811 12:03:07.311549 17649 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:03:07.399180 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 17450
I20260811 12:03:07.423521 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 17664
Aug 11 12:03:07 dist-test-slave-mjjr krb5kdc[17069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449784, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.131@KRBTEST.COM
I20260811 12:03:07.439230 17891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:07.424328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44432 (local address 127.9.221.131:42885)
0811 12:03:07.424532 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:07.424539 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:07.424618 (+ 79us) server_negotiation.cc:407] Connection header received
0811 12:03:07.424836 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:07.424846 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:07.424909 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:07.425027 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:07.426280 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:07.427552 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:07.428932 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:07.429361 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:07.429502 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:07.433866 (+ 4364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:07.433881 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:07.433897 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:07.433931 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:07.436526 (+ 2595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:07.437155 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:07.437163 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:07.437167 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:07.437314 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:07.437736 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:07.437741 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:07.437745 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:07.438098 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:07.438394 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:07.438689 (+ 295us) server_negotiation.cc:299] Negotiation successful
0811 12:03:07.438917 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 12:03:07.439906 17863 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:03:07.528257 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 17664
I20260811 12:03:07.555285 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 17088
I20260811 12:03:07.556512 17195 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:03:07.666074 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 17088
2026-08-11T12:03:07Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4322 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-11T12:03:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:03:07Z Disabled control of system clock
I20260811 12:03:07.745152 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:46569
--webserver_interface=127.9.221.190
--webserver_port=0
--builtin_ntp_servers=127.9.221.148:39223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:46569
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 12:03:08.071146 17952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:03:08.071451 17952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:03:08.071539 17952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:03:08.078784 17952 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:03:08.078912 17952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:03:08.078951 17952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:03:08.079010 17952 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:03:08.079053 17952 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:03:08.088482 17952 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:39223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:46569
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:46569
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.221.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:03:08.089782 17952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:03:08.091707 17952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:03:08.103696 17960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:08.103662 17964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:08.103772 17961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:03:08.109042 17952 server_base.cc:1034] running on GCE node
I20260811 12:03:08.109810 17952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:03:08.111150 17952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:03:08.112396 17952 hybrid_clock.cc:648] HybridClock initialized: now 1786449788112358 us; error 48 us; skew 500 ppm
I20260811 12:03:08.112751 17952 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:03:08.114943 17952 webserver.cc:460] Webserver started at http://127.9.221.190:35157/ using document root <none> and password file <none>
I20260811 12:03:08.115615 17952 fs_manager.cc:359] Metadata directory not provided
I20260811 12:03:08.115814 17952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:03:08.116147 17952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:03:08.118896 17952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "b5a5df20d6494c98969e843f8e408b70"
format_stamp: "Formatted at 2026-08-11 12:03:08 on dist-test-slave-mjjr"
server_key: "614490883331b3712bbb9dcb2b8e8558"
server_key_iv: "a127d2b62220c10a775e164502e83124"
server_key_version: "encryptionkey@0"
I20260811 12:03:08.119733 17952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "b5a5df20d6494c98969e843f8e408b70"
format_stamp: "Formatted at 2026-08-11 12:03:08 on dist-test-slave-mjjr"
server_key: "614490883331b3712bbb9dcb2b8e8558"
server_key_iv: "a127d2b62220c10a775e164502e83124"
server_key_version: "encryptionkey@0"
I20260811 12:03:08.124668 17952 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260811 12:03:08.127880 17973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:08.129060 17952 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 12:03:08.129241 17952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "b5a5df20d6494c98969e843f8e408b70"
format_stamp: "Formatted at 2026-08-11 12:03:08 on dist-test-slave-mjjr"
server_key: "614490883331b3712bbb9dcb2b8e8558"
server_key_iv: "a127d2b62220c10a775e164502e83124"
server_key_version: "encryptionkey@0"
I20260811 12:03:08.129446 17952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:03:08.135128 17952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:03:08.135918 17952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:03:08.136212 17952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:03:08.154458 17952 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.190:46569
I20260811 12:03:08.154543 18049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.190:46569 every 8 connection(s)
I20260811 12:03:08.155756 17952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:03:08.159312 18051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:03:08.162853 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 17952
I20260811 12:03:08.163100 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:03:08.163977 10102 external_mini_cluster.cc:1428] Setting key 4b6ebaa2191b995b0191b7e101a4af72
I20260811 12:03:08.172185 18054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:08.166093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60298 (local address 127.9.221.190:46569)
0811 12:03:08.167109 (+ 1016us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:08.167119 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:08.167158 (+ 39us) server_negotiation.cc:407] Connection header received
0811 12:03:08.168752 (+ 1594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:08.168771 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:08.169169 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:08.169478 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:08.170570 (+ 1092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:08.170576 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:08.170593 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:03:08.170606 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:08.170662 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:03:08.170677 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:08.170882 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:08.171016 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 12:03:08.171259 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":407,"threads_started":1}
I20260811 12:03:08.172948 18051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5a5df20d6494c98969e843f8e408b70: Bootstrap starting.
I20260811 12:03:08.176469 18051 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5a5df20d6494c98969e843f8e408b70: Neither blocks nor log segments found. Creating new log.
I20260811 12:03:08.177421 18051 log.cc:824] T 00000000000000000000000000000000 P b5a5df20d6494c98969e843f8e408b70: Log is configured to *not* fsync() on all Append() calls
I20260811 12:03:08.180198 18051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5a5df20d6494c98969e843f8e408b70: No bootstrap required, opened a new log
I20260811 12:03:08.187098 18051 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5a5df20d6494c98969e843f8e408b70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5a5df20d6494c98969e843f8e408b70" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 46569 } }
I20260811 12:03:08.187393 18051 raft_consensus.cc:374] T 00000000000000000000000000000000 P b5a5df20d6494c98969e843f8e408b70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:03:08.187465 18051 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5a5df20d6494c98969e843f8e408b70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5a5df20d6494c98969e843f8e408b70, State: Initialized, Role: FOLLOWER
I20260811 12:03:08.188014 18051 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5a5df20d6494c98969e843f8e408b70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5a5df20d6494c98969e843f8e408b70" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 46569 } }
I20260811 12:03:08.188215 18051 raft_consensus.cc:388] T 00000000000000000000000000000000 P b5a5df20d6494c98969e843f8e408b70 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:03:08.188308 18051 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5a5df20d6494c98969e843f8e408b70 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:03:08.188432 18051 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5a5df20d6494c98969e843f8e408b70 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:03:08.190392 18051 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5a5df20d6494c98969e843f8e408b70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5a5df20d6494c98969e843f8e408b70" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 46569 } }
I20260811 12:03:08.190788 18051 leader_election.cc:302] T 00000000000000000000000000000000 P b5a5df20d6494c98969e843f8e408b70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b5a5df20d6494c98969e843f8e408b70; no voters:
I20260811 12:03:08.191264 18051 leader_election.cc:288] T 00000000000000000000000000000000 P b5a5df20d6494c98969e843f8e408b70 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:03:08.191413 18057 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5a5df20d6494c98969e843f8e408b70 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:03:08.192195 18057 raft_consensus.cc:686] T 00000000000000000000000000000000 P b5a5df20d6494c98969e843f8e408b70 [term 1 LEADER]: Becoming Leader. State: Replica: b5a5df20d6494c98969e843f8e408b70, State: Running, Role: LEADER
I20260811 12:03:08.192763 18057 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5a5df20d6494c98969e843f8e408b70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5a5df20d6494c98969e843f8e408b70" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 46569 } }
I20260811 12:03:08.193535 18051 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5a5df20d6494c98969e843f8e408b70 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:03:08.195643 18059 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5a5df20d6494c98969e843f8e408b70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5a5df20d6494c98969e843f8e408b70. Latest consensus state: current_term: 1 leader_uuid: "b5a5df20d6494c98969e843f8e408b70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5a5df20d6494c98969e843f8e408b70" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 46569 } } }
I20260811 12:03:08.195777 18058 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5a5df20d6494c98969e843f8e408b70 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b5a5df20d6494c98969e843f8e408b70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5a5df20d6494c98969e843f8e408b70" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 46569 } } }
I20260811 12:03:08.196074 18059 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5a5df20d6494c98969e843f8e408b70 [sys.catalog]: This master's current role is: LEADER
I20260811 12:03:08.196081 18058 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5a5df20d6494c98969e843f8e408b70 [sys.catalog]: This master's current role is: LEADER
I20260811 12:03:08.197722 18072 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:03:08.203045 18072 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:03:08.211246 18072 catalog_manager.cc:1347] Generated new cluster ID: 3b3a299b68e842b39b47f9ac9673f893
I20260811 12:03:08.211362 18072 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:03:08.227433 18072 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:03:08.227881 18072 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:03:08.238236 18072 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b5a5df20d6494c98969e843f8e408b70: Generated new TSK 0
I20260811 12:03:08.238895 18072 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:03:08.245365 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.129:0
--local_ip_for_outbound_sockets=127.9.221.129
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:46569
--builtin_ntp_servers=127.9.221.148:39223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 12:03:08.595098 18089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:03:08.595384 18089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:03:08.595474 18089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:03:08.602787 18089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:03:08.602882 18089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:03:08.602991 18089 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.129
I20260811 12:03:08.612108 18089 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:39223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:46569
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.129
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:03:08.613366 18089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:03:08.615137 18089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:03:08.628296 18095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:08.628379 18096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:08.628201 18098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:03:08.629135 18089 server_base.cc:1034] running on GCE node
I20260811 12:03:08.629701 18089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:03:08.630566 18089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:03:08.631781 18089 hybrid_clock.cc:648] HybridClock initialized: now 1786449788631697 us; error 53 us; skew 500 ppm
I20260811 12:03:08.632131 18089 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:03:08.634668 18089 webserver.cc:460] Webserver started at http://127.9.221.129:35881/ using document root <none> and password file <none>
I20260811 12:03:08.635375 18089 fs_manager.cc:359] Metadata directory not provided
I20260811 12:03:08.635497 18089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:03:08.636003 18089 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:03:08.638955 18089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-0/data/instance:
uuid: "99295cdec1d144ebbf3c8014414f110e"
format_stamp: "Formatted at 2026-08-11 12:03:08 on dist-test-slave-mjjr"
server_key: "129cf2f5ab39f92715cb52345b68b5ee"
server_key_iv: "52eb7840653f92c4aabbed6d65799cea"
server_key_version: "encryptionkey@0"
I20260811 12:03:08.639927 18089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance:
uuid: "99295cdec1d144ebbf3c8014414f110e"
format_stamp: "Formatted at 2026-08-11 12:03:08 on dist-test-slave-mjjr"
server_key: "129cf2f5ab39f92715cb52345b68b5ee"
server_key_iv: "52eb7840653f92c4aabbed6d65799cea"
server_key_version: "encryptionkey@0"
I20260811 12:03:08.644873 18089 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260811 12:03:08.648417 18105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:08.649801 18089 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:03:08.649969 18089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-0/wal
uuid: "99295cdec1d144ebbf3c8014414f110e"
format_stamp: "Formatted at 2026-08-11 12:03:08 on dist-test-slave-mjjr"
server_key: "129cf2f5ab39f92715cb52345b68b5ee"
server_key_iv: "52eb7840653f92c4aabbed6d65799cea"
server_key_version: "encryptionkey@0"
I20260811 12:03:08.650194 18089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:03:08.655990 18089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:03:08.656780 18089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:03:08.657050 18089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:03:08.657948 18089 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:03:08.659417 18089 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:03:08.659529 18089 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:08.659724 18089 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:03:08.659826 18089 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260811 12:03:08.697510 18089 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.129:40603
I20260811 12:03:08.697597 18287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.129:40603 every 8 connection(s)
I20260811 12:03:08.698916 18089 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
I20260811 12:03:08.700278 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 18089
I20260811 12:03:08.700524 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance
I20260811 12:03:08.701288 10102 external_mini_cluster.cc:1428] Setting key 38b6d8df8113d30d3fe1781e71429fc4
I20260811 12:03:08.704797 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.130:0
--local_ip_for_outbound_sockets=127.9.221.130
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:46569
--builtin_ntp_servers=127.9.221.148:39223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 12:03:08.708698 18295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:08.701781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:57395 (local address 127.9.221.190:46569)
0811 12:03:08.702294 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:08.702299 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:08.703768 (+ 1469us) server_negotiation.cc:407] Connection header received
0811 12:03:08.706382 (+ 2614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:08.706386 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:08.706437 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:08.706543 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:08.708046 (+ 1503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:08.708050 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:08.708054 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:03:08.708069 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:08.708111 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:03:08.708122 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:08.708220 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:08.708438 (+ 218us) server_negotiation.cc:299] Negotiation successful
0811 12:03:08.708488 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":259,"threads_started":1}
I20260811 12:03:08.709992 18296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:08.702444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:46569 (local address 127.9.221.129:57395)
0811 12:03:08.703542 (+ 1098us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:08.703616 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:08.706089 (+ 2473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:08.706798 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:08.706812 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:08.706942 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:08.707672 (+ 730us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:03:08.707678 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:08.707731 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:03:08.707845 (+ 114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:08.708219 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:08.708231 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:08.708242 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 12:03:08.708514 (+ 272us) client_negotiation.cc:240] Negotiation successful
0811 12:03:08.708761 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":318,"threads_started":1}
I20260811 12:03:08.711539 18288 heartbeater.cc:347] Connected to a master server at 127.9.221.190:46569
I20260811 12:03:08.711860 18288 heartbeater.cc:464] Registering TS with master...
I20260811 12:03:08.712447 18288 heartbeater.cc:511] Master 127.9.221.190:46569 requested a full tablet report, sending...
I20260811 12:03:08.714601 17996 ts_manager.cc:194] Registered new tserver with Master: 99295cdec1d144ebbf3c8014414f110e (127.9.221.129:40603)
W20260811 12:03:09.037290 18297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:03:09.037616 18297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:03:09.037806 18297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:03:09.044873 18297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:03:09.045007 18297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:03:09.045192 18297 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.130
I20260811 12:03:09.054039 18297 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:39223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:46569
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.130
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:03:09.055382 18297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:03:09.057192 18297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:03:09.070660 18307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:09.070744 18304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:09.070720 18305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:03:09.072948 18297 server_base.cc:1034] running on GCE node
I20260811 12:03:09.073529 18297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:03:09.074420 18297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:03:09.075620 18297 hybrid_clock.cc:648] HybridClock initialized: now 1786449789075580 us; error 27 us; skew 500 ppm
I20260811 12:03:09.075989 18297 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:03:09.078229 18297 webserver.cc:460] Webserver started at http://127.9.221.130:36501/ using document root <none> and password file <none>
I20260811 12:03:09.078922 18297 fs_manager.cc:359] Metadata directory not provided
I20260811 12:03:09.079049 18297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:03:09.079420 18297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:03:09.082342 18297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-1/data/instance:
uuid: "b38bc8f3f7344c14856fe3a0bf258c87"
format_stamp: "Formatted at 2026-08-11 12:03:09 on dist-test-slave-mjjr"
server_key: "7c54112ebfd9311ff24a2551af43580a"
server_key_iv: "654a5f1f92669a8c7d55a10a8f0540fd"
server_key_version: "encryptionkey@0"
I20260811 12:03:09.083133 18297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance:
uuid: "b38bc8f3f7344c14856fe3a0bf258c87"
format_stamp: "Formatted at 2026-08-11 12:03:09 on dist-test-slave-mjjr"
server_key: "7c54112ebfd9311ff24a2551af43580a"
server_key_iv: "654a5f1f92669a8c7d55a10a8f0540fd"
server_key_version: "encryptionkey@0"
I20260811 12:03:09.088210 18297 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 12:03:09.091526 18315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:09.092823 18297 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:03:09.093020 18297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-1/wal
uuid: "b38bc8f3f7344c14856fe3a0bf258c87"
format_stamp: "Formatted at 2026-08-11 12:03:09 on dist-test-slave-mjjr"
server_key: "7c54112ebfd9311ff24a2551af43580a"
server_key_iv: "654a5f1f92669a8c7d55a10a8f0540fd"
server_key_version: "encryptionkey@0"
I20260811 12:03:09.093238 18297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:03:09.099013 18297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:03:09.099836 18297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:03:09.100109 18297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:03:09.101016 18297 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:03:09.102451 18297 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:03:09.102574 18297 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:09.102712 18297 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:03:09.102802 18297 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:09.135468 18297 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.130:42315
I20260811 12:03:09.135521 18501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.130:42315 every 8 connection(s)
I20260811 12:03:09.136786 18297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
I20260811 12:03:09.139038 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 18297
I20260811 12:03:09.139254 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance
I20260811 12:03:09.139789 10102 external_mini_cluster.cc:1428] Setting key 567e3b0495f31b35d8600f7b85697220
I20260811 12:03:09.142715 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.131:0
--local_ip_for_outbound_sockets=127.9.221.131
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:46569
--builtin_ntp_servers=127.9.221.148:39223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 12:03:09.145224 18295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:09.139318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:41733 (local address 127.9.221.190:46569)
0811 12:03:09.139887 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:09.139896 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:09.141201 (+ 1305us) server_negotiation.cc:407] Connection header received
0811 12:03:09.142850 (+ 1649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:09.142859 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:09.142930 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:09.143041 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:09.144396 (+ 1355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:09.144404 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:09.144412 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:03:09.144435 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:09.144489 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:03:09.144506 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:09.144636 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:09.144801 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 12:03:09.144869 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384}
I20260811 12:03:09.146018 18504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:09.139982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:46569 (local address 127.9.221.130:41733)
0811 12:03:09.141031 (+ 1049us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:09.141075 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:09.142577 (+ 1502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:09.143244 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:09.143252 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:09.143334 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:09.144095 (+ 761us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:03:09.144098 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:09.144139 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:03:09.144219 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:09.144610 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:09.144618 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:09.144623 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 12:03:09.144824 (+ 201us) client_negotiation.cc:240] Negotiation successful
0811 12:03:09.145013 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":339,"threads_started":1}
I20260811 12:03:09.147413 18502 heartbeater.cc:347] Connected to a master server at 127.9.221.190:46569
I20260811 12:03:09.147763 18502 heartbeater.cc:464] Registering TS with master...
I20260811 12:03:09.148329 18502 heartbeater.cc:511] Master 127.9.221.190:46569 requested a full tablet report, sending...
I20260811 12:03:09.149760 17996 ts_manager.cc:194] Registered new tserver with Master: b38bc8f3f7344c14856fe3a0bf258c87 (127.9.221.130:42315)
W20260811 12:03:09.476946 18507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:03:09.477190 18507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:03:09.477363 18507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:03:09.484458 18507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:03:09.484549 18507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:03:09.484731 18507 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.131
I20260811 12:03:09.494068 18507 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:39223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:46569
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.131
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:03:09.495405 18507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:03:09.497413 18507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:03:09.511162 18517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:09.511237 18515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:09.511739 18521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:03:09.512887 18507 server_base.cc:1034] running on GCE node
I20260811 12:03:09.513463 18507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:03:09.514361 18507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:03:09.515612 18507 hybrid_clock.cc:648] HybridClock initialized: now 1786449789515567 us; error 61 us; skew 500 ppm
I20260811 12:03:09.515986 18507 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:03:09.518308 18507 webserver.cc:460] Webserver started at http://127.9.221.131:43985/ using document root <none> and password file <none>
I20260811 12:03:09.518986 18507 fs_manager.cc:359] Metadata directory not provided
I20260811 12:03:09.519126 18507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:03:09.519490 18507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:03:09.522321 18507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-2/data/instance:
uuid: "b96646fba7e14587803d15e1de8c498d"
format_stamp: "Formatted at 2026-08-11 12:03:09 on dist-test-slave-mjjr"
server_key: "f20e1cf619152b36bd2807b7a83dce2a"
server_key_iv: "5faba0dae005ad6f889f83237baf95c4"
server_key_version: "encryptionkey@0"
I20260811 12:03:09.523123 18507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance:
uuid: "b96646fba7e14587803d15e1de8c498d"
format_stamp: "Formatted at 2026-08-11 12:03:09 on dist-test-slave-mjjr"
server_key: "f20e1cf619152b36bd2807b7a83dce2a"
server_key_iv: "5faba0dae005ad6f889f83237baf95c4"
server_key_version: "encryptionkey@0"
I20260811 12:03:09.528112 18507 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260811 12:03:09.531409 18529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:09.532697 18507 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:03:09.532871 18507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-2/wal
uuid: "b96646fba7e14587803d15e1de8c498d"
format_stamp: "Formatted at 2026-08-11 12:03:09 on dist-test-slave-mjjr"
server_key: "f20e1cf619152b36bd2807b7a83dce2a"
server_key_iv: "5faba0dae005ad6f889f83237baf95c4"
server_key_version: "encryptionkey@0"
I20260811 12:03:09.533104 18507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:03:09.538546 18507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:03:09.539333 18507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:03:09.539626 18507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:03:09.540567 18507 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:03:09.541976 18507 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:03:09.542105 18507 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:09.542248 18507 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:03:09.542339 18507 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:09.575508 18507 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.131:36833
I20260811 12:03:09.575598 18698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.131:36833 every 8 connection(s)
I20260811 12:03:09.576841 18507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
I20260811 12:03:09.578671 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 18507
I20260811 12:03:09.578882 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance
I20260811 12:03:09.579360 10102 external_mini_cluster.cc:1428] Setting key d82436dc333f011c97022d9d8217e400
I20260811 12:03:09.585424 18295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:09.579432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:41531 (local address 127.9.221.190:46569)
0811 12:03:09.579696 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:09.579702 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:09.581389 (+ 1687us) server_negotiation.cc:407] Connection header received
0811 12:03:09.583173 (+ 1784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:09.583179 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:09.583249 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:09.583372 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:09.584666 (+ 1294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:09.584672 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:09.584677 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:03:09.584695 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:09.584737 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:03:09.584749 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:09.584870 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:09.585071 (+ 201us) server_negotiation.cc:299] Negotiation successful
0811 12:03:09.585141 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 12:03:09.586285 18703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:09.580055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:46569 (local address 127.9.221.131:41531)
0811 12:03:09.581184 (+ 1129us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:09.581271 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:09.582909 (+ 1638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:09.583603 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:09.583611 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:09.583760 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:09.584374 (+ 614us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:03:09.584377 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:09.584412 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:03:09.584500 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:09.584845 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:09.584856 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:09.584863 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 12:03:09.585176 (+ 313us) client_negotiation.cc:240] Negotiation successful
0811 12:03:09.585405 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":341,"threads_started":1}
I20260811 12:03:09.587442 18701 heartbeater.cc:347] Connected to a master server at 127.9.221.190:46569
I20260811 12:03:09.587760 18701 heartbeater.cc:464] Registering TS with master...
I20260811 12:03:09.588326 18701 heartbeater.cc:511] Master 127.9.221.190:46569 requested a full tablet report, sending...
I20260811 12:03:09.589624 17996 ts_manager.cc:194] Registered new tserver with Master: b96646fba7e14587803d15e1de8c498d (127.9.221.131:36833)
I20260811 12:03:09.594638 10102 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:03:09.601491 18295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:09.599254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60300 (local address 127.9.221.190:46569)
0811 12:03:09.599382 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:09.599386 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:09.599975 (+ 589us) server_negotiation.cc:407] Connection header received
0811 12:03:09.600108 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:09.600112 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:09.600161 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:09.600236 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:09.600970 (+ 734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:09.600974 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:09.600977 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:03:09.600991 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:09.601017 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:03:09.601028 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:09.601118 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:09.601153 (+ 35us) server_negotiation.cc:299] Negotiation successful
0811 12:03:09.601193 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 12:03:09.605055 17996 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60300:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:03:09.614357 17996 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:03:09.635763 18723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:09.631685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.130:42315 (local address 127.0.0.1:51608)
0811 12:03:09.632233 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:09.632259 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:09.632406 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:09.633402 (+ 996us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:09.633406 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:09.633429 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:09.634413 (+ 984us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:03:09.634417 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:09.634431 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:03:09.634483 (+ 52us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:09.634891 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:09.634903 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:09.634911 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 12:03:09.635085 (+ 174us) client_negotiation.cc:240] Negotiation successful
0811 12:03:09.635189 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":248,"threads_started":1}
I20260811 12:03:09.635926 18726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:09.631954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51608 (local address 127.9.221.130:42315)
0811 12:03:09.632895 (+ 941us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:09.632903 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:09.632930 (+ 27us) server_negotiation.cc:407] Connection header received
0811 12:03:09.633013 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:09.633022 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:09.633250 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:09.633405 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:09.634630 (+ 1225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:09.634640 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:09.634652 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:03:09.634672 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:09.634737 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:03:09.634760 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:09.635405 (+ 645us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:09.635471 (+ 66us) server_negotiation.cc:299] Negotiation successful
0811 12:03:09.635563 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":779,"thread_start_us":661,"threads_started":1}
I20260811 12:03:09.636749 18722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:09.630583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.131:36833 (local address 127.0.0.1:36796)
0811 12:03:09.631281 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:09.631301 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:09.631391 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:09.632049 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:09.632060 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:09.632176 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:09.633223 (+ 1047us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:03:09.633229 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:09.633265 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:03:09.633370 (+ 105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:09.634957 (+ 1587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:09.634965 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:09.634973 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 12:03:09.635143 (+ 170us) client_negotiation.cc:240] Negotiation successful
0811 12:03:09.635209 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":674,"threads_started":1}
I20260811 12:03:09.637970 18719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:09.629430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36796 (local address 127.9.221.131:36833)
0811 12:03:09.631304 (+ 1874us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:09.631313 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:09.631354 (+ 41us) server_negotiation.cc:407] Connection header received
0811 12:03:09.631549 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:09.631558 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:09.631812 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:09.632772 (+ 960us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:09.634662 (+ 1890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:09.634673 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:09.634690 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:03:09.634711 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:09.634772 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:03:09.634795 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:09.637201 (+ 2406us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:09.637281 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 12:03:09.637388 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1702,"thread_start_us":1520,"threads_started":1}
I20260811 12:03:09.639925 18401 tablet_service.cc:1467] Processing CreateTablet for tablet 364158e640ab40488f9888f47b519890 (DEFAULT_TABLE table=test-table [id=2de7e63837fd4b3aa2059e5bafe6f357]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:03:09.641261 18401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 364158e640ab40488f9888f47b519890. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:03:09.642227 18596 tablet_service.cc:1467] Processing CreateTablet for tablet 364158e640ab40488f9888f47b519890 (DEFAULT_TABLE table=test-table [id=2de7e63837fd4b3aa2059e5bafe6f357]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:03:09.643702 18596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 364158e640ab40488f9888f47b519890. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:03:09.647825 18721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:09.630158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42674 (local address 127.9.221.129:40603)
0811 12:03:09.642557 (+ 12399us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:09.642565 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:09.642593 (+ 28us) server_negotiation.cc:407] Connection header received
0811 12:03:09.642659 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:09.642665 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:09.642849 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:09.643037 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:09.646941 (+ 3904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:09.646951 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:09.646963 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:03:09.646993 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:09.647050 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:03:09.647069 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:09.647207 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:09.647375 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 12:03:09.647464 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12222,"thread_start_us":13739,"threads_started":1}
I20260811 12:03:09.647884 18718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:09.629456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:40603 (local address 127.0.0.1:42674)
0811 12:03:09.630256 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:09.630315 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:09.630491 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:09.643043 (+ 12552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:09.643048 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:09.643071 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:09.643797 (+ 726us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:03:09.643801 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:09.643819 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:03:09.643864 (+ 45us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:09.647203 (+ 3339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:09.647208 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:09.647213 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 12:03:09.647369 (+ 156us) client_negotiation.cc:240] Negotiation successful
0811 12:03:09.647427 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":269,"threads_started":1}
I20260811 12:03:09.650928 18182 tablet_service.cc:1467] Processing CreateTablet for tablet 364158e640ab40488f9888f47b519890 (DEFAULT_TABLE table=test-table [id=2de7e63837fd4b3aa2059e5bafe6f357]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:03:09.652264 18182 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 364158e640ab40488f9888f47b519890. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:03:09.657074 18727 tablet_bootstrap.cc:492] T 364158e640ab40488f9888f47b519890 P b96646fba7e14587803d15e1de8c498d: Bootstrap starting.
I20260811 12:03:09.660249 18727 tablet_bootstrap.cc:654] T 364158e640ab40488f9888f47b519890 P b96646fba7e14587803d15e1de8c498d: Neither blocks nor log segments found. Creating new log.
I20260811 12:03:09.661248 18727 log.cc:824] T 364158e640ab40488f9888f47b519890 P b96646fba7e14587803d15e1de8c498d: Log is configured to *not* fsync() on all Append() calls
I20260811 12:03:09.663111 18728 tablet_bootstrap.cc:492] T 364158e640ab40488f9888f47b519890 P b38bc8f3f7344c14856fe3a0bf258c87: Bootstrap starting.
I20260811 12:03:09.664216 18727 tablet_bootstrap.cc:492] T 364158e640ab40488f9888f47b519890 P b96646fba7e14587803d15e1de8c498d: No bootstrap required, opened a new log
I20260811 12:03:09.664475 18727 ts_tablet_manager.cc:1397] T 364158e640ab40488f9888f47b519890 P b96646fba7e14587803d15e1de8c498d: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.001s
I20260811 12:03:09.666355 18728 tablet_bootstrap.cc:654] T 364158e640ab40488f9888f47b519890 P b38bc8f3f7344c14856fe3a0bf258c87: Neither blocks nor log segments found. Creating new log.
I20260811 12:03:09.667603 18728 log.cc:824] T 364158e640ab40488f9888f47b519890 P b38bc8f3f7344c14856fe3a0bf258c87: Log is configured to *not* fsync() on all Append() calls
I20260811 12:03:09.668520 18732 tablet_bootstrap.cc:492] T 364158e640ab40488f9888f47b519890 P 99295cdec1d144ebbf3c8014414f110e: Bootstrap starting.
I20260811 12:03:09.671838 18732 tablet_bootstrap.cc:654] T 364158e640ab40488f9888f47b519890 P 99295cdec1d144ebbf3c8014414f110e: Neither blocks nor log segments found. Creating new log.
I20260811 12:03:09.673022 18732 log.cc:824] T 364158e640ab40488f9888f47b519890 P 99295cdec1d144ebbf3c8014414f110e: Log is configured to *not* fsync() on all Append() calls
I20260811 12:03:09.673514 18727 raft_consensus.cc:348] T 364158e640ab40488f9888f47b519890 P b96646fba7e14587803d15e1de8c498d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96646fba7e14587803d15e1de8c498d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 36833 } } peers { permanent_uuid: "99295cdec1d144ebbf3c8014414f110e" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 40603 } } peers { permanent_uuid: "b38bc8f3f7344c14856fe3a0bf258c87" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 42315 } }
I20260811 12:03:09.673980 18727 raft_consensus.cc:374] T 364158e640ab40488f9888f47b519890 P b96646fba7e14587803d15e1de8c498d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:03:09.674108 18727 raft_consensus.cc:729] T 364158e640ab40488f9888f47b519890 P b96646fba7e14587803d15e1de8c498d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96646fba7e14587803d15e1de8c498d, State: Initialized, Role: FOLLOWER
I20260811 12:03:09.674739 18727 consensus_queue.cc:260] T 364158e640ab40488f9888f47b519890 P b96646fba7e14587803d15e1de8c498d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96646fba7e14587803d15e1de8c498d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 36833 } } peers { permanent_uuid: "99295cdec1d144ebbf3c8014414f110e" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 40603 } } peers { permanent_uuid: "b38bc8f3f7344c14856fe3a0bf258c87" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 42315 } }
I20260811 12:03:09.676455 18732 tablet_bootstrap.cc:492] T 364158e640ab40488f9888f47b519890 P 99295cdec1d144ebbf3c8014414f110e: No bootstrap required, opened a new log
I20260811 12:03:09.676652 18701 heartbeater.cc:503] Master 127.9.221.190:46569 was elected leader, sending a full tablet report...
I20260811 12:03:09.676740 18732 ts_tablet_manager.cc:1397] T 364158e640ab40488f9888f47b519890 P 99295cdec1d144ebbf3c8014414f110e: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260811 12:03:09.676846 18728 tablet_bootstrap.cc:492] T 364158e640ab40488f9888f47b519890 P b38bc8f3f7344c14856fe3a0bf258c87: No bootstrap required, opened a new log
I20260811 12:03:09.677117 18728 ts_tablet_manager.cc:1397] T 364158e640ab40488f9888f47b519890 P b38bc8f3f7344c14856fe3a0bf258c87: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.008s
I20260811 12:03:09.677291 18727 ts_tablet_manager.cc:1428] T 364158e640ab40488f9888f47b519890 P b96646fba7e14587803d15e1de8c498d: Time spent starting tablet: real 0.013s user 0.002s sys 0.010s
I20260811 12:03:09.685038 18728 raft_consensus.cc:348] T 364158e640ab40488f9888f47b519890 P b38bc8f3f7344c14856fe3a0bf258c87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96646fba7e14587803d15e1de8c498d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 36833 } } peers { permanent_uuid: "99295cdec1d144ebbf3c8014414f110e" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 40603 } } peers { permanent_uuid: "b38bc8f3f7344c14856fe3a0bf258c87" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 42315 } }
I20260811 12:03:09.685391 18728 raft_consensus.cc:374] T 364158e640ab40488f9888f47b519890 P b38bc8f3f7344c14856fe3a0bf258c87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:03:09.685173 18732 raft_consensus.cc:348] T 364158e640ab40488f9888f47b519890 P 99295cdec1d144ebbf3c8014414f110e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96646fba7e14587803d15e1de8c498d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 36833 } } peers { permanent_uuid: "99295cdec1d144ebbf3c8014414f110e" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 40603 } } peers { permanent_uuid: "b38bc8f3f7344c14856fe3a0bf258c87" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 42315 } }
I20260811 12:03:09.685586 18732 raft_consensus.cc:374] T 364158e640ab40488f9888f47b519890 P 99295cdec1d144ebbf3c8014414f110e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:03:09.685514 18728 raft_consensus.cc:729] T 364158e640ab40488f9888f47b519890 P b38bc8f3f7344c14856fe3a0bf258c87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b38bc8f3f7344c14856fe3a0bf258c87, State: Initialized, Role: FOLLOWER
I20260811 12:03:09.685689 18732 raft_consensus.cc:729] T 364158e640ab40488f9888f47b519890 P 99295cdec1d144ebbf3c8014414f110e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99295cdec1d144ebbf3c8014414f110e, State: Initialized, Role: FOLLOWER
I20260811 12:03:09.686239 18728 consensus_queue.cc:260] T 364158e640ab40488f9888f47b519890 P b38bc8f3f7344c14856fe3a0bf258c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96646fba7e14587803d15e1de8c498d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 36833 } } peers { permanent_uuid: "99295cdec1d144ebbf3c8014414f110e" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 40603 } } peers { permanent_uuid: "b38bc8f3f7344c14856fe3a0bf258c87" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 42315 } }
I20260811 12:03:09.686252 18732 consensus_queue.cc:260] T 364158e640ab40488f9888f47b519890 P 99295cdec1d144ebbf3c8014414f110e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96646fba7e14587803d15e1de8c498d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 36833 } } peers { permanent_uuid: "99295cdec1d144ebbf3c8014414f110e" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 40603 } } peers { permanent_uuid: "b38bc8f3f7344c14856fe3a0bf258c87" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 42315 } }
I20260811 12:03:09.687023 18735 raft_consensus.cc:482] T 364158e640ab40488f9888f47b519890 P b96646fba7e14587803d15e1de8c498d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:03:09.687282 18288 heartbeater.cc:503] Master 127.9.221.190:46569 was elected leader, sending a full tablet report...
I20260811 12:03:09.687289 18735 raft_consensus.cc:504] T 364158e640ab40488f9888f47b519890 P b96646fba7e14587803d15e1de8c498d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96646fba7e14587803d15e1de8c498d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 36833 } } peers { permanent_uuid: "99295cdec1d144ebbf3c8014414f110e" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 40603 } } peers { permanent_uuid: "b38bc8f3f7344c14856fe3a0bf258c87" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 42315 } }
I20260811 12:03:09.687855 18732 ts_tablet_manager.cc:1428] T 364158e640ab40488f9888f47b519890 P 99295cdec1d144ebbf3c8014414f110e: Time spent starting tablet: real 0.011s user 0.008s sys 0.003s
I20260811 12:03:09.688568 18502 heartbeater.cc:503] Master 127.9.221.190:46569 was elected leader, sending a full tablet report...
I20260811 12:03:09.689097 18735 leader_election.cc:288] T 364158e640ab40488f9888f47b519890 P b96646fba7e14587803d15e1de8c498d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99295cdec1d144ebbf3c8014414f110e (127.9.221.129:40603), b38bc8f3f7344c14856fe3a0bf258c87 (127.9.221.130:42315)
I20260811 12:03:09.689425 18728 ts_tablet_manager.cc:1428] T 364158e640ab40488f9888f47b519890 P b38bc8f3f7344c14856fe3a0bf258c87: Time spent starting tablet: real 0.012s user 0.010s sys 0.000s
I20260811 12:03:09.692803 18703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:09.689698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.130:42315 (local address 127.9.221.131:54811)
0811 12:03:09.690010 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:09.690029 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:09.690129 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:09.690907 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:09.690914 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:09.690938 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:09.691569 (+ 631us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:03:09.691575 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:09.691600 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:03:09.691703 (+ 103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:09.692208 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:09.692216 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:09.692224 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 12:03:09.692337 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 12:03:09.692406 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185}
I20260811 12:03:09.694193 18726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:09.690468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:54811 (local address 127.9.221.130:42315)
0811 12:03:09.690638 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:09.690644 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:09.690661 (+ 17us) server_negotiation.cc:407] Connection header received
0811 12:03:09.690714 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:09.690719 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:09.690768 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:09.690880 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:09.691904 (+ 1024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:09.691913 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:09.691921 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:03:09.691942 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:09.691992 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:03:09.692013 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:09.692155 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:09.693741 (+ 1586us) server_negotiation.cc:299] Negotiation successful
0811 12:03:09.693829 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 12:03:09.695238 18431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "364158e640ab40488f9888f47b519890" candidate_uuid: "b96646fba7e14587803d15e1de8c498d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b38bc8f3f7344c14856fe3a0bf258c87" is_pre_election: true
I20260811 12:03:09.695837 18431 raft_consensus.cc:2454] T 364158e640ab40488f9888f47b519890 P b38bc8f3f7344c14856fe3a0bf258c87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b96646fba7e14587803d15e1de8c498d in term 0.
I20260811 12:03:09.696630 18721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:09.690338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:58379 (local address 127.9.221.129:40603)
0811 12:03:09.690523 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:09.690531 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:09.693311 (+ 2780us) server_negotiation.cc:407] Connection header received
0811 12:03:09.693379 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:09.693387 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:09.693463 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:09.693560 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:09.696004 (+ 2444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:09.696009 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:09.696015 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:03:09.696036 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:09.696070 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:03:09.696084 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:09.696184 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:09.696305 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 12:03:09.696360 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 12:03:09.696668 18703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:09.690317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:40603 (local address 127.9.221.131:58379)
0811 12:03:09.693072 (+ 2755us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:09.693094 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:09.693193 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:09.694466 (+ 1273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:09.694473 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:09.694505 (+ 32us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:09.695791 (+ 1286us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:03:09.695793 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:09.695807 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:03:09.695862 (+ 55us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:09.696164 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:09.696168 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:09.696173 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 12:03:09.696309 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 12:03:09.696356 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2625,"threads_started":1}
I20260811 12:03:09.696745 18532 leader_election.cc:302] T 364158e640ab40488f9888f47b519890 P b96646fba7e14587803d15e1de8c498d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b38bc8f3f7344c14856fe3a0bf258c87, b96646fba7e14587803d15e1de8c498d; no voters:
I20260811 12:03:09.697139 18735 raft_consensus.cc:2781] T 364158e640ab40488f9888f47b519890 P b96646fba7e14587803d15e1de8c498d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:03:09.697292 18735 raft_consensus.cc:482] T 364158e640ab40488f9888f47b519890 P b96646fba7e14587803d15e1de8c498d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:03:09.697404 18735 raft_consensus.cc:3037] T 364158e640ab40488f9888f47b519890 P b96646fba7e14587803d15e1de8c498d [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:03:09.699844 18735 raft_consensus.cc:504] T 364158e640ab40488f9888f47b519890 P b96646fba7e14587803d15e1de8c498d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96646fba7e14587803d15e1de8c498d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 36833 } } peers { permanent_uuid: "99295cdec1d144ebbf3c8014414f110e" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 40603 } } peers { permanent_uuid: "b38bc8f3f7344c14856fe3a0bf258c87" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 42315 } }
I20260811 12:03:09.700372 18735 leader_election.cc:288] T 364158e640ab40488f9888f47b519890 P b96646fba7e14587803d15e1de8c498d [CANDIDATE]: Term 1 election: Requested vote from peers 99295cdec1d144ebbf3c8014414f110e (127.9.221.129:40603), b38bc8f3f7344c14856fe3a0bf258c87 (127.9.221.130:42315)
I20260811 12:03:09.700917 18431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "364158e640ab40488f9888f47b519890" candidate_uuid: "b96646fba7e14587803d15e1de8c498d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b38bc8f3f7344c14856fe3a0bf258c87"
I20260811 12:03:09.701113 18431 raft_consensus.cc:3037] T 364158e640ab40488f9888f47b519890 P b38bc8f3f7344c14856fe3a0bf258c87 [term 0 FOLLOWER]: Advancing to term 1
W20260811 12:03:09.701256 18290 tablet.cc:2369] T 364158e640ab40488f9888f47b519890 P 99295cdec1d144ebbf3c8014414f110e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:03:09.703352 18431 raft_consensus.cc:2454] T 364158e640ab40488f9888f47b519890 P b38bc8f3f7344c14856fe3a0bf258c87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96646fba7e14587803d15e1de8c498d in term 1.
I20260811 12:03:09.703806 18532 leader_election.cc:302] T 364158e640ab40488f9888f47b519890 P b96646fba7e14587803d15e1de8c498d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b38bc8f3f7344c14856fe3a0bf258c87, b96646fba7e14587803d15e1de8c498d; no voters:
I20260811 12:03:09.704699 18735 raft_consensus.cc:2781] T 364158e640ab40488f9888f47b519890 P b96646fba7e14587803d15e1de8c498d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:03:09.705230 18735 raft_consensus.cc:686] T 364158e640ab40488f9888f47b519890 P b96646fba7e14587803d15e1de8c498d [term 1 LEADER]: Becoming Leader. State: Replica: b96646fba7e14587803d15e1de8c498d, State: Running, Role: LEADER
I20260811 12:03:09.705724 18735 consensus_queue.cc:237] T 364158e640ab40488f9888f47b519890 P b96646fba7e14587803d15e1de8c498d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96646fba7e14587803d15e1de8c498d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 36833 } } peers { permanent_uuid: "99295cdec1d144ebbf3c8014414f110e" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 40603 } } peers { permanent_uuid: "b38bc8f3f7344c14856fe3a0bf258c87" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 42315 } }
I20260811 12:03:09.706110 18211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "364158e640ab40488f9888f47b519890" candidate_uuid: "b96646fba7e14587803d15e1de8c498d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99295cdec1d144ebbf3c8014414f110e"
I20260811 12:03:09.706156 18212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "364158e640ab40488f9888f47b519890" candidate_uuid: "b96646fba7e14587803d15e1de8c498d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99295cdec1d144ebbf3c8014414f110e" is_pre_election: true
I20260811 12:03:09.706576 18211 raft_consensus.cc:3037] T 364158e640ab40488f9888f47b519890 P 99295cdec1d144ebbf3c8014414f110e [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:03:09.709894 18211 raft_consensus.cc:2454] T 364158e640ab40488f9888f47b519890 P 99295cdec1d144ebbf3c8014414f110e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b96646fba7e14587803d15e1de8c498d in term 1.
I20260811 12:03:09.713027 17994 catalog_manager.cc:5495] T 364158e640ab40488f9888f47b519890 P b96646fba7e14587803d15e1de8c498d reported cstate change: term changed from 0 to 1, leader changed from <none> to b96646fba7e14587803d15e1de8c498d (127.9.221.131). New cstate: current_term: 1 leader_uuid: "b96646fba7e14587803d15e1de8c498d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b96646fba7e14587803d15e1de8c498d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 36833 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99295cdec1d144ebbf3c8014414f110e" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 40603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b38bc8f3f7344c14856fe3a0bf258c87" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 42315 } health_report { overall_health: UNKNOWN } } }
I20260811 12:03:09.738968 18719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:09.736820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36798 (local address 127.9.221.131:36833)
0811 12:03:09.736997 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:09.737004 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:09.737189 (+ 185us) server_negotiation.cc:407] Connection header received
0811 12:03:09.737377 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:09.737382 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:09.737480 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:09.737566 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:09.738409 (+ 843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:09.738412 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:09.738415 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:03:09.738430 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:09.738461 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:03:09.738472 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:09.738570 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:09.738633 (+ 63us) server_negotiation.cc:299] Negotiation successful
0811 12:03:09.738675 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 12:03:09.746790 18211 raft_consensus.cc:1261] T 364158e640ab40488f9888f47b519890 P 99295cdec1d144ebbf3c8014414f110e [term 1 FOLLOWER]: Refusing update from remote peer b96646fba7e14587803d15e1de8c498d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:03:09.746881 18431 raft_consensus.cc:1261] T 364158e640ab40488f9888f47b519890 P b38bc8f3f7344c14856fe3a0bf258c87 [term 1 FOLLOWER]: Refusing update from remote peer b96646fba7e14587803d15e1de8c498d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:03:09.747754 18741 consensus_queue.cc:1035] T 364158e640ab40488f9888f47b519890 P b96646fba7e14587803d15e1de8c498d [LEADER]: Connected to new peer: Peer: permanent_uuid: "99295cdec1d144ebbf3c8014414f110e" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 40603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:03:09.748147 18735 consensus_queue.cc:1035] T 364158e640ab40488f9888f47b519890 P b96646fba7e14587803d15e1de8c498d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b38bc8f3f7344c14856fe3a0bf258c87" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 42315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:03:09.759606 18746 mvcc.cc:205] Tried to move back new op lower bound from 7317298338792660992 to 7317298338639368192. Current Snapshot: MvccSnapshot[applied={T|T < 7317298338792660992}]
I20260811 12:03:09.759912 18747 mvcc.cc:205] Tried to move back new op lower bound from 7317298338792660992 to 7317298338639368192. Current Snapshot: MvccSnapshot[applied={T|T < 7317298338792660992}]
I20260811 12:03:09.772145 17996 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:03:09.772419 17996 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:03:09.776075 17996 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5a5df20d6494c98969e843f8e408b70: Sending DeleteTablet for 3 replicas of tablet 364158e640ab40488f9888f47b519890
I20260811 12:03:09.777318 18596 tablet_service.cc:1514] Processing DeleteTablet for tablet 364158e640ab40488f9888f47b519890 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:03:09 UTC) from {username='slave'} at 127.0.0.1:36796
I20260811 12:03:09.777298 18182 tablet_service.cc:1514] Processing DeleteTablet for tablet 364158e640ab40488f9888f47b519890 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:03:09 UTC) from {username='slave'} at 127.0.0.1:42674
I20260811 12:03:09.778323 18401 tablet_service.cc:1514] Processing DeleteTablet for tablet 364158e640ab40488f9888f47b519890 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:03:09 UTC) from {username='slave'} at 127.0.0.1:51608
I20260811 12:03:09.778923 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 18089
I20260811 12:03:09.793577 18721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:09.780346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42682 (local address 127.9.221.129:40603)
0811 12:03:09.780559 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:09.780567 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:09.790765 (+ 10198us) server_negotiation.cc:407] Connection header received
0811 12:03:09.791229 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:09.791235 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:09.791350 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:09.791440 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:09.792748 (+ 1308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:09.792754 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:09.792759 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:03:09.792778 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:09.792815 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:03:09.792830 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:09.792921 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:09.793167 (+ 246us) server_negotiation.cc:299] Negotiation successful
0811 12:03:09.793249 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 12:03:09.794243 18761 tablet_replica.cc:330] T 364158e640ab40488f9888f47b519890 P b38bc8f3f7344c14856fe3a0bf258c87: stopping tablet replica
I20260811 12:03:09.794657 18281 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:03:09.794719 18761 raft_consensus.cc:2229] T 364158e640ab40488f9888f47b519890 P b38bc8f3f7344c14856fe3a0bf258c87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:03:09.795262 18761 raft_consensus.cc:2258] T 364158e640ab40488f9888f47b519890 P b38bc8f3f7344c14856fe3a0bf258c87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:03:09.796185 18759 tablet_replica.cc:330] T 364158e640ab40488f9888f47b519890 P b96646fba7e14587803d15e1de8c498d: stopping tablet replica
I20260811 12:03:09.796595 18759 raft_consensus.cc:2229] T 364158e640ab40488f9888f47b519890 P b96646fba7e14587803d15e1de8c498d [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:03:09.797392 18759 raft_consensus.cc:2258] T 364158e640ab40488f9888f47b519890 P b96646fba7e14587803d15e1de8c498d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:03:09.799796 18761 ts_tablet_manager.cc:1905] T 364158e640ab40488f9888f47b519890 P b38bc8f3f7344c14856fe3a0bf258c87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:03:09.805436 18759 ts_tablet_manager.cc:1905] T 364158e640ab40488f9888f47b519890 P b96646fba7e14587803d15e1de8c498d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:03:09.812656 18761 ts_tablet_manager.cc:1918] T 364158e640ab40488f9888f47b519890 P b38bc8f3f7344c14856fe3a0bf258c87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:03:09.812844 18759 ts_tablet_manager.cc:1918] T 364158e640ab40488f9888f47b519890 P b96646fba7e14587803d15e1de8c498d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:03:09.812903 18761 log.cc:1196] T 364158e640ab40488f9888f47b519890 P b38bc8f3f7344c14856fe3a0bf258c87: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-1/wal/wals/364158e640ab40488f9888f47b519890
I20260811 12:03:09.813006 18759 log.cc:1196] T 364158e640ab40488f9888f47b519890 P b96646fba7e14587803d15e1de8c498d: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449743084027-10102-0/minicluster-data/ts-2/wal/wals/364158e640ab40488f9888f47b519890
I20260811 12:03:09.813570 18759 ts_tablet_manager.cc:1939] T 364158e640ab40488f9888f47b519890 P b96646fba7e14587803d15e1de8c498d: Deleting consensus metadata
I20260811 12:03:09.813609 18761 ts_tablet_manager.cc:1939] T 364158e640ab40488f9888f47b519890 P b38bc8f3f7344c14856fe3a0bf258c87: Deleting consensus metadata
I20260811 12:03:09.815472 17977 catalog_manager.cc:4841] TS b96646fba7e14587803d15e1de8c498d (127.9.221.131:36833): tablet 364158e640ab40488f9888f47b519890 (table test-table [id=2de7e63837fd4b3aa2059e5bafe6f357]) successfully deleted
I20260811 12:03:09.815873 17977 catalog_manager.cc:4841] TS b38bc8f3f7344c14856fe3a0bf258c87 (127.9.221.130:42315): tablet 364158e640ab40488f9888f47b519890 (table test-table [id=2de7e63837fd4b3aa2059e5bafe6f357]) successfully deleted
I20260811 12:03:09.905565 18758 tablet_replica.cc:330] T 364158e640ab40488f9888f47b519890 P 99295cdec1d144ebbf3c8014414f110e: stopping tablet replica
I20260811 12:03:09.905912 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 18089
I20260811 12:03:09.929682 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 18297
W20260811 12:03:09.929713 17978 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.221.129:40603 (error 108)
I20260811 12:03:09.931813 18718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:09.931268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:40603 (local address 127.0.0.1:42696)
0811 12:03:09.931480 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:09.931627 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.129:40603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260811 12:03:09.932099 17978 catalog_manager.cc:4584] TS 99295cdec1d144ebbf3c8014414f110e (127.9.221.129:40603): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 364158e640ab40488f9888f47b519890: Network error: Client connection negotiation failed: client connection to 127.9.221.129:40603: connect: Connection refused (error 111)
I20260811 12:03:09.933174 18726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:09.930979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51616 (local address 127.9.221.130:42315)
0811 12:03:09.931149 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:09.931153 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:09.931170 (+ 17us) server_negotiation.cc:407] Connection header received
0811 12:03:09.931242 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:09.931246 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:09.931379 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:09.931490 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:09.932531 (+ 1041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:09.932536 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:09.932539 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:03:09.932554 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:09.932616 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:03:09.932631 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:09.932742 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:09.932836 (+ 94us) server_negotiation.cc:299] Negotiation successful
0811 12:03:09.932881 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 12:03:09.933878 18495 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:03:09.986212 18718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:09.985625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:40603 (local address 127.0.0.1:42698)
0811 12:03:09.985878 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:09.986001 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.129:40603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260811 12:03:09.987432 18718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:09.987030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:40603 (local address 127.0.0.1:42710)
0811 12:03:09.987198 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:09.987293 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.129:40603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 12:03:10.021381 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 18297
I20260811 12:03:10.048379 18718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:10.047793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:40603 (local address 127.0.0.1:42722)
0811 12:03:10.048008 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:10.048166 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.129:40603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 12:03:10.048454 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 18507
I20260811 12:03:10.049782 18718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:10.049263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:40603 (local address 127.0.0.1:42736)
0811 12:03:10.049530 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:10.049621 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.129:40603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162}
I20260811 12:03:10.051851 18719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:10.049258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36808 (local address 127.9.221.131:36833)
0811 12:03:10.049483 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:10.049488 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:10.049725 (+ 237us) server_negotiation.cc:407] Connection header received
0811 12:03:10.050049 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:10.050053 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:10.050119 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:10.050232 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:10.051158 (+ 926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:10.051164 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:10.051169 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:03:10.051191 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:10.051221 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:03:10.051232 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:10.051376 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:10.051475 (+ 99us) server_negotiation.cc:299] Negotiation successful
0811 12:03:10.051536 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260811 12:03:10.052621 18691 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:03:10.130357 18718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:10.129861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:40603 (local address 127.0.0.1:42746)
0811 12:03:10.130082 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:10.130180 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.129:40603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260811 12:03:10.131466 18718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:10.131046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:40603 (local address 127.0.0.1:42752)
0811 12:03:10.131267 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:10.131339 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.129:40603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260811 12:03:10.170894 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 18507
I20260811 12:03:10.196660 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 17952
I20260811 12:03:10.197796 18041 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:03:10.305063 18718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:10.304585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:40603 (local address 127.0.0.1:42764)
0811 12:03:10.304811 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:10.304899 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.129:40603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260811 12:03:10.305107 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 17952
2026-08-11T12:03:10Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2632 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260811 12:03:10.350662 10102 mini_oidc.cc:152] Starting JWKS server
I20260811 12:03:10.353631 10102 webserver.cc:460] Webserver started at https://127.0.0.1:46015/ using document root <none> and password file <none>
I20260811 12:03:10.353973 10102 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 12:03:10.354595 10102 webserver.cc:460] Webserver started at http://127.0.0.1:45373/ using document root <none> and password file <none>
2026-08-11T12:03:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:03:10Z Disabled control of system clock
I20260811 12:03:10.384200 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:40749
--webserver_interface=127.9.221.190
--webserver_port=0
--builtin_ntp_servers=127.9.221.148:41549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:40749
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46015/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786449743084027-10102-0/testchainca.cert with env {}
W20260811 12:03:10.709620 18777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:03:10.709932 18777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:03:10.710095 18777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:03:10.716971 18777 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:03:10.717099 18777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:03:10.717186 18777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:03:10.717316 18777 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:03:10.717403 18777 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 12:03:10.717492 18777 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 12:03:10.717572 18777 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46015/jwks.json
I20260811 12:03:10.726866 18777 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:41549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:40749
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:40749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46015/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.221.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786449743084027-10102-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:03:10.728045 18777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:03:10.729306 18777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:03:10.740906 18783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:10.740849 18785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:10.740950 18787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:03:10.741745 18777 server_base.cc:1034] running on GCE node
I20260811 12:03:10.742491 18777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:03:10.743774 18777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:03:10.744998 18777 hybrid_clock.cc:648] HybridClock initialized: now 1786449790744962 us; error 47 us; skew 500 ppm
I20260811 12:03:10.745325 18777 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:03:10.752647 18777 webserver.cc:460] Webserver started at http://127.9.221.190:43627/ using document root <none> and password file <none>
I20260811 12:03:10.753306 18777 fs_manager.cc:359] Metadata directory not provided
I20260811 12:03:10.753448 18777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:03:10.753746 18777 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:03:10.756654 18777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "56dee11b505c48ecb46b3bbcf23b7f1f"
format_stamp: "Formatted at 2026-08-11 12:03:10 on dist-test-slave-mjjr"
server_key: "b24dd79f34316299c039385f1d74f6ee"
server_key_iv: "e2dd7977d95d846b71a235ecefe1c7b9"
server_key_version: "encryptionkey@0"
I20260811 12:03:10.757406 18777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "56dee11b505c48ecb46b3bbcf23b7f1f"
format_stamp: "Formatted at 2026-08-11 12:03:10 on dist-test-slave-mjjr"
server_key: "b24dd79f34316299c039385f1d74f6ee"
server_key_iv: "e2dd7977d95d846b71a235ecefe1c7b9"
server_key_version: "encryptionkey@0"
I20260811 12:03:10.762375 18777 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:03:10.765782 18799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:10.767071 18777 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:03:10.767246 18777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "56dee11b505c48ecb46b3bbcf23b7f1f"
format_stamp: "Formatted at 2026-08-11 12:03:10 on dist-test-slave-mjjr"
server_key: "b24dd79f34316299c039385f1d74f6ee"
server_key_iv: "e2dd7977d95d846b71a235ecefe1c7b9"
server_key_version: "encryptionkey@0"
I20260811 12:03:10.767462 18777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:03:10.782204 18777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:03:10.783197 18777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:03:10.783564 18777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:03:10.802030 18777 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.190:40749
I20260811 12:03:10.802068 18887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.190:40749 every 8 connection(s)
I20260811 12:03:10.803290 18777 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:03:10.807052 18888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:03:10.811445 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 18777
I20260811 12:03:10.811671 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:03:10.812319 10102 external_mini_cluster.cc:1428] Setting key 9867fdb51e1b48b3ea131275375edcc4
I20260811 12:03:10.817785 18888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56dee11b505c48ecb46b3bbcf23b7f1f: Bootstrap starting.
I20260811 12:03:10.821368 18888 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56dee11b505c48ecb46b3bbcf23b7f1f: Neither blocks nor log segments found. Creating new log.
I20260811 12:03:10.822594 18888 log.cc:824] T 00000000000000000000000000000000 P 56dee11b505c48ecb46b3bbcf23b7f1f: Log is configured to *not* fsync() on all Append() calls
I20260811 12:03:10.825686 18888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56dee11b505c48ecb46b3bbcf23b7f1f: No bootstrap required, opened a new log
I20260811 12:03:10.826117 18892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:10.814180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38434 (local address 127.9.221.190:40749)
0811 12:03:10.815202 (+ 1022us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:10.815211 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:10.815253 (+ 42us) server_negotiation.cc:407] Connection header received
0811 12:03:10.816996 (+ 1743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:10.817016 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:10.817418 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:10.817714 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:10.818386 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:10.819707 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:10.821633 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:10.822289 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:10.822419 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:10.823823 (+ 1404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:10.823836 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:10.823856 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:03:10.824441 (+ 585us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:10.824511 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:03:10.824535 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:10.824619 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:10.824867 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 12:03:10.825198 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":399,"threads_started":1}
I20260811 12:03:10.829734 10102 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:03:10.833807 18888 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56dee11b505c48ecb46b3bbcf23b7f1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56dee11b505c48ecb46b3bbcf23b7f1f" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40749 } }
I20260811 12:03:10.834136 18888 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56dee11b505c48ecb46b3bbcf23b7f1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:03:10.834203 18888 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56dee11b505c48ecb46b3bbcf23b7f1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56dee11b505c48ecb46b3bbcf23b7f1f, State: Initialized, Role: FOLLOWER
I20260811 12:03:10.834740 18888 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56dee11b505c48ecb46b3bbcf23b7f1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56dee11b505c48ecb46b3bbcf23b7f1f" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40749 } }
I20260811 12:03:10.834956 18888 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56dee11b505c48ecb46b3bbcf23b7f1f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:03:10.835104 18888 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56dee11b505c48ecb46b3bbcf23b7f1f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:03:10.835294 18888 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56dee11b505c48ecb46b3bbcf23b7f1f [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:03:10.837411 18888 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56dee11b505c48ecb46b3bbcf23b7f1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56dee11b505c48ecb46b3bbcf23b7f1f" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40749 } }
I20260811 12:03:10.837898 18888 leader_election.cc:302] T 00000000000000000000000000000000 P 56dee11b505c48ecb46b3bbcf23b7f1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56dee11b505c48ecb46b3bbcf23b7f1f; no voters:
I20260811 12:03:10.838416 18888 leader_election.cc:288] T 00000000000000000000000000000000 P 56dee11b505c48ecb46b3bbcf23b7f1f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:03:10.838572 18894 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56dee11b505c48ecb46b3bbcf23b7f1f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:03:10.839525 18894 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56dee11b505c48ecb46b3bbcf23b7f1f [term 1 LEADER]: Becoming Leader. State: Replica: 56dee11b505c48ecb46b3bbcf23b7f1f, State: Running, Role: LEADER
I20260811 12:03:10.840023 18894 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56dee11b505c48ecb46b3bbcf23b7f1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56dee11b505c48ecb46b3bbcf23b7f1f" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40749 } }
I20260811 12:03:10.840616 18888 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56dee11b505c48ecb46b3bbcf23b7f1f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:03:10.842847 18898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56dee11b505c48ecb46b3bbcf23b7f1f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56dee11b505c48ecb46b3bbcf23b7f1f. Latest consensus state: current_term: 1 leader_uuid: "56dee11b505c48ecb46b3bbcf23b7f1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56dee11b505c48ecb46b3bbcf23b7f1f" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40749 } } }
I20260811 12:03:10.842927 18896 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56dee11b505c48ecb46b3bbcf23b7f1f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "56dee11b505c48ecb46b3bbcf23b7f1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56dee11b505c48ecb46b3bbcf23b7f1f" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40749 } } }
I20260811 12:03:10.843360 18898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56dee11b505c48ecb46b3bbcf23b7f1f [sys.catalog]: This master's current role is: LEADER
I20260811 12:03:10.843360 18896 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56dee11b505c48ecb46b3bbcf23b7f1f [sys.catalog]: This master's current role is: LEADER
I20260811 12:03:10.845046 18901 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:03:10.849836 18901 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:03:10.857712 18901 catalog_manager.cc:1347] Generated new cluster ID: 23115dd333674cbdb388c590628c9e0a
I20260811 12:03:10.857812 18901 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:03:10.869552 18901 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:03:10.871338 18901 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:03:10.878615 18901 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56dee11b505c48ecb46b3bbcf23b7f1f: Generated new TSK 0
I20260811 12:03:10.879530 18901 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 12:03:10.932221 18929 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260811 12:03:10.932443 18929 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 12:03:10.934384 18892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:10.905083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38436 (local address 127.9.221.190:40749)
0811 12:03:10.905284 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:10.905294 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:10.905339 (+ 45us) server_negotiation.cc:407] Connection header received
0811 12:03:10.905812 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:10.905836 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:10.905940 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:10.906098 (+ 158us) server_negotiation.cc:226] Negotiated authn=JWT
0811 12:03:10.906677 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:10.907410 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:10.908972 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:10.909461 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:10.909613 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:10.910166 (+ 553us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 12:03:10.933655 (+ 23489us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46015/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 12:03:10.934060 (+ 405us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38436: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46015/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":68}
W20260811 12:03:10.934558 18892 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38436: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46015/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260811 12:03:10.936483 18928 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:10.904578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:40749 (local address 127.0.0.1:38436)
0811 12:03:10.905246 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:10.905267 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:10.905459 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:10.906131 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:10.906136 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:10.906152 (+ 16us) client_negotiation.cc:189] Negotiated authn=JWT
0811 12:03:10.906520 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:10.906529 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:10.907570 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:10.907575 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:10.908813 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:10.908825 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:10.909671 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:10.909680 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:10.909827 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:10.909885 (+ 58us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 12:03:10.933978 (+ 24093us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46015/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 12:03:10.934339 (+ 361us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.221.190:40749: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46015/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":549,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.9.221.190:40749: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46015/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260811 12:03:10.938402 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 18777
I20260811 12:03:10.939544 18881 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:03:11.031388 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 18777
2026-08-11T12:03:11Z chronyd exiting
I20260811 12:03:11.088536 10102 test_util.cc:175] -----------------------------------------------
I20260811 12:03:11.088687 10102 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786449743084027-10102-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (742 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260811 12:03:11.092512 10102 mini_oidc.cc:152] Starting JWKS server
I20260811 12:03:11.094352 10102 webserver.cc:460] Webserver started at https://127.0.0.1:36065/ using document root <none> and password file <none>
I20260811 12:03:11.094612 10102 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 12:03:11.095281 10102 webserver.cc:460] Webserver started at http://127.0.0.1:43243/ using document root <none> and password file <none>
2026-08-11T12:03:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:03:11Z Disabled control of system clock
I20260811 12:03:11.126020 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:33527
--webserver_interface=127.9.221.190
--webserver_port=0
--builtin_ntp_servers=127.9.221.148:33775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:33527
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36065/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786449743084027-10102-0/testchainca.cert with env {}
W20260811 12:03:11.452315 18940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:03:11.452625 18940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:03:11.452788 18940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:03:11.459436 18940 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:03:11.459539 18940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:03:11.459573 18940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:03:11.459635 18940 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:03:11.459746 18940 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 12:03:11.459831 18940 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 12:03:11.459872 18940 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36065/jwks.json
I20260811 12:03:11.469841 18940 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:33775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:33527
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:33527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36065/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.221.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786449743084027-10102-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:03:11.470985 18940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:03:11.472275 18940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:03:11.484272 18950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:11.484311 18947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:11.484357 18948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:03:11.484890 18940 server_base.cc:1034] running on GCE node
I20260811 12:03:11.485694 18940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:03:11.486994 18940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:03:11.488220 18940 hybrid_clock.cc:648] HybridClock initialized: now 1786449791488132 us; error 75 us; skew 500 ppm
I20260811 12:03:11.488559 18940 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:03:11.490687 18940 webserver.cc:460] Webserver started at http://127.9.221.190:44781/ using document root <none> and password file <none>
I20260811 12:03:11.491340 18940 fs_manager.cc:359] Metadata directory not provided
I20260811 12:03:11.491478 18940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:03:11.491870 18940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:03:11.494658 18940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "24ca2ab07ee84484a588feb8f99483c0"
format_stamp: "Formatted at 2026-08-11 12:03:11 on dist-test-slave-mjjr"
server_key: "6032560069e33f33272a59a3fe080282"
server_key_iv: "3f760e34c94f5181f27518184161dea8"
server_key_version: "encryptionkey@0"
I20260811 12:03:11.495447 18940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "24ca2ab07ee84484a588feb8f99483c0"
format_stamp: "Formatted at 2026-08-11 12:03:11 on dist-test-slave-mjjr"
server_key: "6032560069e33f33272a59a3fe080282"
server_key_iv: "3f760e34c94f5181f27518184161dea8"
server_key_version: "encryptionkey@0"
I20260811 12:03:11.500427 18940 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260811 12:03:11.503746 18958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:11.505086 18940 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 12:03:11.505263 18940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "24ca2ab07ee84484a588feb8f99483c0"
format_stamp: "Formatted at 2026-08-11 12:03:11 on dist-test-slave-mjjr"
server_key: "6032560069e33f33272a59a3fe080282"
server_key_iv: "3f760e34c94f5181f27518184161dea8"
server_key_version: "encryptionkey@0"
I20260811 12:03:11.505489 18940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:03:11.526126 18940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:03:11.526898 18940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:03:11.527211 18940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:03:11.545578 18940 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.190:33527
I20260811 12:03:11.545596 19040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.190:33527 every 8 connection(s)
I20260811 12:03:11.546911 18940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:03:11.550486 19041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:03:11.554548 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 18940
I20260811 12:03:11.554772 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:03:11.555603 10102 external_mini_cluster.cc:1428] Setting key 4a187c2a43c915190d007389d42228a8
I20260811 12:03:11.561791 19041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24ca2ab07ee84484a588feb8f99483c0: Bootstrap starting.
I20260811 12:03:11.565287 19041 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24ca2ab07ee84484a588feb8f99483c0: Neither blocks nor log segments found. Creating new log.
I20260811 12:03:11.566584 19041 log.cc:824] T 00000000000000000000000000000000 P 24ca2ab07ee84484a588feb8f99483c0: Log is configured to *not* fsync() on all Append() calls
I20260811 12:03:11.569772 19041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24ca2ab07ee84484a588feb8f99483c0: No bootstrap required, opened a new log
I20260811 12:03:11.569867 19048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:11.557499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57490 (local address 127.9.221.190:33527)
0811 12:03:11.558473 (+ 974us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:11.558483 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:11.558523 (+ 40us) server_negotiation.cc:407] Connection header received
0811 12:03:11.560486 (+ 1963us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:11.560507 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:11.560911 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:11.561208 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:11.561805 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:11.563161 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:11.564984 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:11.565767 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:11.565898 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:11.567338 (+ 1440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:11.567354 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:11.567372 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:03:11.568186 (+ 814us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:11.568267 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:03:11.568287 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:11.568375 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:11.568618 (+ 243us) server_negotiation.cc:299] Negotiation successful
0811 12:03:11.568934 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":382,"threads_started":1}
I20260811 12:03:11.573717 10102 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:03:11.577986 19041 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24ca2ab07ee84484a588feb8f99483c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24ca2ab07ee84484a588feb8f99483c0" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 33527 } }
I20260811 12:03:11.578338 19041 raft_consensus.cc:374] T 00000000000000000000000000000000 P 24ca2ab07ee84484a588feb8f99483c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:03:11.578442 19041 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24ca2ab07ee84484a588feb8f99483c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24ca2ab07ee84484a588feb8f99483c0, State: Initialized, Role: FOLLOWER
I20260811 12:03:11.578985 19041 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24ca2ab07ee84484a588feb8f99483c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24ca2ab07ee84484a588feb8f99483c0" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 33527 } }
I20260811 12:03:11.579227 19041 raft_consensus.cc:388] T 00000000000000000000000000000000 P 24ca2ab07ee84484a588feb8f99483c0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:03:11.579370 19041 raft_consensus.cc:482] T 00000000000000000000000000000000 P 24ca2ab07ee84484a588feb8f99483c0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:03:11.579541 19041 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24ca2ab07ee84484a588feb8f99483c0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:03:11.581681 19041 raft_consensus.cc:504] T 00000000000000000000000000000000 P 24ca2ab07ee84484a588feb8f99483c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24ca2ab07ee84484a588feb8f99483c0" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 33527 } }
I20260811 12:03:11.582175 19041 leader_election.cc:302] T 00000000000000000000000000000000 P 24ca2ab07ee84484a588feb8f99483c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 24ca2ab07ee84484a588feb8f99483c0; no voters:
I20260811 12:03:11.582829 19041 leader_election.cc:288] T 00000000000000000000000000000000 P 24ca2ab07ee84484a588feb8f99483c0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:03:11.583029 19050 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 24ca2ab07ee84484a588feb8f99483c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:03:11.583766 19050 raft_consensus.cc:686] T 00000000000000000000000000000000 P 24ca2ab07ee84484a588feb8f99483c0 [term 1 LEADER]: Becoming Leader. State: Replica: 24ca2ab07ee84484a588feb8f99483c0, State: Running, Role: LEADER
I20260811 12:03:11.584311 19050 consensus_queue.cc:237] T 00000000000000000000000000000000 P 24ca2ab07ee84484a588feb8f99483c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24ca2ab07ee84484a588feb8f99483c0" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 33527 } }
I20260811 12:03:11.584893 19041 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24ca2ab07ee84484a588feb8f99483c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:03:11.587029 19051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24ca2ab07ee84484a588feb8f99483c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "24ca2ab07ee84484a588feb8f99483c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24ca2ab07ee84484a588feb8f99483c0" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 33527 } } }
I20260811 12:03:11.587484 19051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24ca2ab07ee84484a588feb8f99483c0 [sys.catalog]: This master's current role is: LEADER
I20260811 12:03:11.587872 19053 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24ca2ab07ee84484a588feb8f99483c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24ca2ab07ee84484a588feb8f99483c0. Latest consensus state: current_term: 1 leader_uuid: "24ca2ab07ee84484a588feb8f99483c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24ca2ab07ee84484a588feb8f99483c0" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 33527 } } }
I20260811 12:03:11.590341 19053 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24ca2ab07ee84484a588feb8f99483c0 [sys.catalog]: This master's current role is: LEADER
I20260811 12:03:11.594734 19062 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:03:11.600073 19062 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:03:11.608057 19062 catalog_manager.cc:1347] Generated new cluster ID: cc569968dc5649098e6f401bfa5a4367
I20260811 12:03:11.608165 19062 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:03:11.640694 19062 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:03:11.642532 19062 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:03:11.654847 19062 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 24ca2ab07ee84484a588feb8f99483c0: Generated new TSK 0
I20260811 12:03:11.655854 19062 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 12:03:11.687430 19085 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260811 12:03:11.687556 19085 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 12:03:11.688761 19048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:11.668696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57502 (local address 127.9.221.190:33527)
0811 12:03:11.668925 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:11.668934 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:11.669499 (+ 565us) server_negotiation.cc:407] Connection header received
0811 12:03:11.669711 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:11.669716 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:11.669841 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:11.669997 (+ 156us) server_negotiation.cc:226] Negotiated authn=JWT
0811 12:03:11.670408 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:11.671499 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:11.672851 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:11.673513 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:11.673645 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:11.674069 (+ 424us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 12:03:11.688102 (+ 14033us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36065/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 12:03:11.688473 (+ 371us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57502: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36065/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":46}
W20260811 12:03:11.688782 19084 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:11.668526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:33527 (local address 127.0.0.1:57502)
0811 12:03:11.669394 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:11.669410 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:11.669524 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:11.670013 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:11.670016 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:11.670025 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0811 12:03:11.670256 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:11.670263 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:11.671620 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:11.671623 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:11.672713 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:11.672725 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:11.673673 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:11.673677 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:11.673809 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:11.673838 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 12:03:11.688298 (+ 14460us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36065/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 12:03:11.688477 (+ 179us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.221.190:33527: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36065/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":738,"thread_start_us":606,"threads_started":1}
W20260811 12:03:11.688892 19048 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57502: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36065/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260811 12:03:11.690248 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 18940
I20260811 12:03:11.691340 19033 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:03:11.784881 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 18940
2026-08-11T12:03:11Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (758 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260811 12:03:11.851015 10102 mini_oidc.cc:152] Starting JWKS server
I20260811 12:03:11.853045 10102 webserver.cc:460] Webserver started at https://127.0.0.1:42163/ using document root <none> and password file <none>
I20260811 12:03:11.853279 10102 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 12:03:11.853921 10102 webserver.cc:460] Webserver started at http://127.0.0.1:39267/ using document root <none> and password file <none>
2026-08-11T12:03:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:03:11Z Disabled control of system clock
I20260811 12:03:11.886236 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:33227
--webserver_interface=127.9.221.190
--webserver_port=0
--builtin_ntp_servers=127.9.221.148:33897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:33227
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42163/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 12:03:12.209457 19095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:03:12.209777 19095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:03:12.209937 19095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:03:12.216898 19095 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:03:12.217026 19095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:03:12.217118 19095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:03:12.217204 19095 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:03:12.217267 19095 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 12:03:12.217350 19095 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 12:03:12.217437 19095 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42163/jwks.json
I20260811 12:03:12.226282 19095 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:33897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:33227
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:33227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42163/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.221.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:03:12.227540 19095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:03:12.229324 19095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:03:12.240599 19102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:12.240622 19103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:12.240653 19105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:03:12.241657 19095 server_base.cc:1034] running on GCE node
I20260811 12:03:12.242379 19095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:03:12.243746 19095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:03:12.244973 19095 hybrid_clock.cc:648] HybridClock initialized: now 1786449792244936 us; error 48 us; skew 500 ppm
I20260811 12:03:12.245368 19095 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:03:12.247485 19095 webserver.cc:460] Webserver started at http://127.9.221.190:36109/ using document root <none> and password file <none>
I20260811 12:03:12.248202 19095 fs_manager.cc:359] Metadata directory not provided
I20260811 12:03:12.248353 19095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:03:12.248672 19095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:03:12.251422 19095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "6e9ae35f4cd14d499a63e35334e8af4e"
format_stamp: "Formatted at 2026-08-11 12:03:12 on dist-test-slave-mjjr"
server_key: "9505a51393a132b3423479012845989c"
server_key_iv: "85929f8aba6719b27543e629bf634bcf"
server_key_version: "encryptionkey@0"
I20260811 12:03:12.252281 19095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "6e9ae35f4cd14d499a63e35334e8af4e"
format_stamp: "Formatted at 2026-08-11 12:03:12 on dist-test-slave-mjjr"
server_key: "9505a51393a132b3423479012845989c"
server_key_iv: "85929f8aba6719b27543e629bf634bcf"
server_key_version: "encryptionkey@0"
I20260811 12:03:12.257192 19095 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 12:03:12.260483 19117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:12.261781 19095 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:03:12.261988 19095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "6e9ae35f4cd14d499a63e35334e8af4e"
format_stamp: "Formatted at 2026-08-11 12:03:12 on dist-test-slave-mjjr"
server_key: "9505a51393a132b3423479012845989c"
server_key_iv: "85929f8aba6719b27543e629bf634bcf"
server_key_version: "encryptionkey@0"
I20260811 12:03:12.262207 19095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:03:12.281179 19095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:03:12.282021 19095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:03:12.282341 19095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:03:12.301257 19095 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.190:33227
I20260811 12:03:12.301293 19207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.190:33227 every 8 connection(s)
I20260811 12:03:12.302546 19095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:03:12.306305 19209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:03:12.306375 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 19095
I20260811 12:03:12.306591 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:03:12.307466 10102 external_mini_cluster.cc:1428] Setting key bf2f8f39b98b1899681e532b026fb2b6
I20260811 12:03:12.320313 19209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e9ae35f4cd14d499a63e35334e8af4e: Bootstrap starting.
I20260811 12:03:12.321604 19214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:12.309410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41428 (local address 127.9.221.190:33227)
0811 12:03:12.310459 (+ 1049us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:12.310470 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:12.310513 (+ 43us) server_negotiation.cc:407] Connection header received
0811 12:03:12.312689 (+ 2176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:12.312708 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:12.313119 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:12.313436 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:12.314012 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:12.315199 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:12.317148 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:12.317927 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:12.318050 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:12.319338 (+ 1288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:12.319345 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:12.319357 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:03:12.319954 (+ 597us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:12.320027 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:03:12.320053 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:12.320135 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:12.320379 (+ 244us) server_negotiation.cc:299] Negotiation successful
0811 12:03:12.320738 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":646,"thread_start_us":433,"threads_started":1}
I20260811 12:03:12.323509 19209 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e9ae35f4cd14d499a63e35334e8af4e: Neither blocks nor log segments found. Creating new log.
I20260811 12:03:12.324728 19209 log.cc:824] T 00000000000000000000000000000000 P 6e9ae35f4cd14d499a63e35334e8af4e: Log is configured to *not* fsync() on all Append() calls
I20260811 12:03:12.325182 10102 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:03:12.327961 19209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e9ae35f4cd14d499a63e35334e8af4e: No bootstrap required, opened a new log
I20260811 12:03:12.335896 19209 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e9ae35f4cd14d499a63e35334e8af4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e9ae35f4cd14d499a63e35334e8af4e" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 33227 } }
I20260811 12:03:12.336273 19209 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e9ae35f4cd14d499a63e35334e8af4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:03:12.336417 19209 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e9ae35f4cd14d499a63e35334e8af4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e9ae35f4cd14d499a63e35334e8af4e, State: Initialized, Role: FOLLOWER
I20260811 12:03:12.336947 19209 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e9ae35f4cd14d499a63e35334e8af4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e9ae35f4cd14d499a63e35334e8af4e" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 33227 } }
I20260811 12:03:12.337194 19209 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e9ae35f4cd14d499a63e35334e8af4e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:03:12.337344 19209 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e9ae35f4cd14d499a63e35334e8af4e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:03:12.337525 19209 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e9ae35f4cd14d499a63e35334e8af4e [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:03:12.339632 19209 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e9ae35f4cd14d499a63e35334e8af4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e9ae35f4cd14d499a63e35334e8af4e" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 33227 } }
I20260811 12:03:12.340428 19209 leader_election.cc:302] T 00000000000000000000000000000000 P 6e9ae35f4cd14d499a63e35334e8af4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e9ae35f4cd14d499a63e35334e8af4e; no voters:
I20260811 12:03:12.341042 19209 leader_election.cc:288] T 00000000000000000000000000000000 P 6e9ae35f4cd14d499a63e35334e8af4e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:03:12.341168 19217 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e9ae35f4cd14d499a63e35334e8af4e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:03:12.341945 19217 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e9ae35f4cd14d499a63e35334e8af4e [term 1 LEADER]: Becoming Leader. State: Replica: 6e9ae35f4cd14d499a63e35334e8af4e, State: Running, Role: LEADER
I20260811 12:03:12.342482 19217 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e9ae35f4cd14d499a63e35334e8af4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e9ae35f4cd14d499a63e35334e8af4e" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 33227 } }
I20260811 12:03:12.343183 19209 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e9ae35f4cd14d499a63e35334e8af4e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:03:12.345323 19220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e9ae35f4cd14d499a63e35334e8af4e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e9ae35f4cd14d499a63e35334e8af4e. Latest consensus state: current_term: 1 leader_uuid: "6e9ae35f4cd14d499a63e35334e8af4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e9ae35f4cd14d499a63e35334e8af4e" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 33227 } } }
I20260811 12:03:12.345788 19220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e9ae35f4cd14d499a63e35334e8af4e [sys.catalog]: This master's current role is: LEADER
I20260811 12:03:12.347497 19219 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e9ae35f4cd14d499a63e35334e8af4e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e9ae35f4cd14d499a63e35334e8af4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e9ae35f4cd14d499a63e35334e8af4e" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 33227 } } }
I20260811 12:03:12.347893 19219 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e9ae35f4cd14d499a63e35334e8af4e [sys.catalog]: This master's current role is: LEADER
I20260811 12:03:12.349128 19228 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:03:12.353478 19228 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:03:12.362111 19228 catalog_manager.cc:1347] Generated new cluster ID: f7f5d872b05445c2a28dd5493b8bcc32
I20260811 12:03:12.362228 19228 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:03:12.394286 19228 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:03:12.396204 19228 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:03:12.407765 19228 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e9ae35f4cd14d499a63e35334e8af4e: Generated new TSK 0
I20260811 12:03:12.408805 19228 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:03:12.433136 19214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:12.424483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41430 (local address 127.9.221.190:33227)
0811 12:03:12.424721 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:12.424731 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:12.425156 (+ 425us) server_negotiation.cc:407] Connection header received
0811 12:03:12.425387 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:12.425391 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:12.425466 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:12.425587 (+ 121us) server_negotiation.cc:226] Negotiated authn=JWT
0811 12:03:12.426152 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:12.426880 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:12.428902 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:12.429360 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:12.429456 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:12.430083 (+ 627us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 12:03:12.432524 (+ 2441us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42163/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 12:03:12.432827 (+ 303us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41430: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42163/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":68}
W20260811 12:03:12.433257 19214 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41430: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42163/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260811 12:03:12.435343 19249 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:12.424389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:33227 (local address 127.0.0.1:41430)
0811 12:03:12.425047 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:12.425066 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:12.425226 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:12.425635 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:12.425641 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:12.425655 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0811 12:03:12.425981 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:12.425993 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:12.427039 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:12.427045 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:12.428763 (+ 1718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:12.428778 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:12.429561 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:12.429568 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:12.429758 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:12.429804 (+ 46us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 12:03:12.432733 (+ 2929us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42163/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 12:03:12.434869 (+ 2136us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.221.190:33227: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42163/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":505,"mutex_wait_us":1818,"thread_start_us":338,"threads_started":1}
I20260811 12:03:12.436976 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 19095
I20260811 12:03:12.438177 19201 generic_service.cc:196] Checking for leaks (request via RPC)
E20260811 12:03:12.445545 19252 webserver.cc:533] Webserver: sslize failed:
E20260811 12:03:12.445623 19252 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 12:03:12.531975 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 19095
2026-08-11T12:03:12Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (745 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-11T12:03:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:03:12Z Disabled control of system clock
I20260811 12:03:12.628634 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:34115
--webserver_interface=127.9.221.190
--webserver_port=0
--builtin_ntp_servers=127.9.221.148:39603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:34115
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786449743084027-10102-0/testchainca.cert with env {}
W20260811 12:03:12.954176 19259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:03:12.954478 19259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:03:12.954641 19259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:03:12.961969 19259 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:03:12.962087 19259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:03:12.962143 19259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:03:12.962208 19259 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:03:12.962297 19259 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 12:03:12.962414 19259 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 12:03:12.962482 19259 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260811 12:03:12.972185 19259 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:39603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:34115
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:34115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.221.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786449743084027-10102-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:03:12.973554 19259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:03:12.974884 19259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:03:12.986572 19264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:12.986631 19266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:12.986851 19268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:03:12.987470 19259 server_base.cc:1034] running on GCE node
I20260811 12:03:12.988297 19259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:03:12.989665 19259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:03:12.990900 19259 hybrid_clock.cc:648] HybridClock initialized: now 1786449792990858 us; error 54 us; skew 500 ppm
I20260811 12:03:12.991266 19259 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:03:12.993474 19259 webserver.cc:460] Webserver started at http://127.9.221.190:35249/ using document root <none> and password file <none>
I20260811 12:03:12.994124 19259 fs_manager.cc:359] Metadata directory not provided
I20260811 12:03:12.994271 19259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:03:12.994587 19259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:03:12.997552 19259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "7da15bd125ed468197f4b37598b639b8"
format_stamp: "Formatted at 2026-08-11 12:03:12 on dist-test-slave-mjjr"
server_key: "6fe4970935e2d4019e169198ea99cb5e"
server_key_iv: "832837149251aa90fb165b4ebb25da1e"
server_key_version: "encryptionkey@0"
I20260811 12:03:12.998375 19259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "7da15bd125ed468197f4b37598b639b8"
format_stamp: "Formatted at 2026-08-11 12:03:12 on dist-test-slave-mjjr"
server_key: "6fe4970935e2d4019e169198ea99cb5e"
server_key_iv: "832837149251aa90fb165b4ebb25da1e"
server_key_version: "encryptionkey@0"
I20260811 12:03:13.003285 19259 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 12:03:13.006587 19278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:13.007866 19259 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:03:13.008046 19259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "7da15bd125ed468197f4b37598b639b8"
format_stamp: "Formatted at 2026-08-11 12:03:12 on dist-test-slave-mjjr"
server_key: "6fe4970935e2d4019e169198ea99cb5e"
server_key_iv: "832837149251aa90fb165b4ebb25da1e"
server_key_version: "encryptionkey@0"
I20260811 12:03:13.008315 19259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:03:13.022634 19259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:03:13.023414 19259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:03:13.023717 19259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:03:13.041813 19259 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.190:34115
I20260811 12:03:13.041832 19369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.190:34115 every 8 connection(s)
I20260811 12:03:13.043079 19259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:03:13.046612 19370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:03:13.047077 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 19259
I20260811 12:03:13.047300 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:03:13.048004 10102 external_mini_cluster.cc:1428] Setting key 45cebd231fc8fe2bb43cbbb2c0b3e174
I20260811 12:03:13.059810 19370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7da15bd125ed468197f4b37598b639b8: Bootstrap starting.
I20260811 12:03:13.061543 19375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:13.049800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36314 (local address 127.9.221.190:34115)
0811 12:03:13.050891 (+ 1091us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:13.050901 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:13.050941 (+ 40us) server_negotiation.cc:407] Connection header received
0811 12:03:13.052660 (+ 1719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:13.052681 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:13.053137 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:13.053461 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:13.054044 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:13.055224 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:13.057211 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:13.057932 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:13.058059 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:13.059270 (+ 1211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:13.059277 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:13.059288 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:03:13.059866 (+ 578us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:13.059977 (+ 111us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:03:13.060006 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:13.060099 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:13.060334 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 12:03:13.060653 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":699,"thread_start_us":490,"threads_started":1}
I20260811 12:03:13.063180 19370 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7da15bd125ed468197f4b37598b639b8: Neither blocks nor log segments found. Creating new log.
I20260811 12:03:13.064353 19370 log.cc:824] T 00000000000000000000000000000000 P 7da15bd125ed468197f4b37598b639b8: Log is configured to *not* fsync() on all Append() calls
I20260811 12:03:13.065281 10102 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:03:13.066884 19370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7da15bd125ed468197f4b37598b639b8: No bootstrap required, opened a new log
I20260811 12:03:13.073793 19370 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7da15bd125ed468197f4b37598b639b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7da15bd125ed468197f4b37598b639b8" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 34115 } }
I20260811 12:03:13.074194 19370 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7da15bd125ed468197f4b37598b639b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:03:13.074324 19370 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7da15bd125ed468197f4b37598b639b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7da15bd125ed468197f4b37598b639b8, State: Initialized, Role: FOLLOWER
I20260811 12:03:13.074885 19370 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7da15bd125ed468197f4b37598b639b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7da15bd125ed468197f4b37598b639b8" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 34115 } }
I20260811 12:03:13.075129 19370 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7da15bd125ed468197f4b37598b639b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:03:13.075263 19370 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7da15bd125ed468197f4b37598b639b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:03:13.075433 19370 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7da15bd125ed468197f4b37598b639b8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:03:13.077531 19370 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7da15bd125ed468197f4b37598b639b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7da15bd125ed468197f4b37598b639b8" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 34115 } }
I20260811 12:03:13.078015 19370 leader_election.cc:302] T 00000000000000000000000000000000 P 7da15bd125ed468197f4b37598b639b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7da15bd125ed468197f4b37598b639b8; no voters:
I20260811 12:03:13.078707 19370 leader_election.cc:288] T 00000000000000000000000000000000 P 7da15bd125ed468197f4b37598b639b8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:03:13.078830 19378 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7da15bd125ed468197f4b37598b639b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:03:13.079564 19378 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7da15bd125ed468197f4b37598b639b8 [term 1 LEADER]: Becoming Leader. State: Replica: 7da15bd125ed468197f4b37598b639b8, State: Running, Role: LEADER
I20260811 12:03:13.080148 19378 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7da15bd125ed468197f4b37598b639b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7da15bd125ed468197f4b37598b639b8" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 34115 } }
I20260811 12:03:13.080731 19370 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7da15bd125ed468197f4b37598b639b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:03:13.083004 19380 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7da15bd125ed468197f4b37598b639b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7da15bd125ed468197f4b37598b639b8. Latest consensus state: current_term: 1 leader_uuid: "7da15bd125ed468197f4b37598b639b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7da15bd125ed468197f4b37598b639b8" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 34115 } } }
I20260811 12:03:13.083738 19380 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7da15bd125ed468197f4b37598b639b8 [sys.catalog]: This master's current role is: LEADER
I20260811 12:03:13.083439 19379 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7da15bd125ed468197f4b37598b639b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7da15bd125ed468197f4b37598b639b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7da15bd125ed468197f4b37598b639b8" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 34115 } } }
I20260811 12:03:13.083958 19379 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7da15bd125ed468197f4b37598b639b8 [sys.catalog]: This master's current role is: LEADER
I20260811 12:03:13.089212 19387 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:03:13.094321 19387 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:03:13.103543 19387 catalog_manager.cc:1347] Generated new cluster ID: e5611e03275c44a28a10b7dbeed664bd
I20260811 12:03:13.103689 19387 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:03:13.137897 19387 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:03:13.139219 19387 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:03:13.146227 19387 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7da15bd125ed468197f4b37598b639b8: Generated new TSK 0
I20260811 12:03:13.147018 19387 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:03:13.185636 19375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:13.165684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36318 (local address 127.9.221.190:34115)
0811 12:03:13.165906 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:13.165917 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:13.166438 (+ 521us) server_negotiation.cc:407] Connection header received
0811 12:03:13.166679 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:13.166683 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:13.166749 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:13.166867 (+ 118us) server_negotiation.cc:226] Negotiated authn=JWT
0811 12:03:13.167414 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:13.168185 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:13.170228 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:13.170655 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:13.170742 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:13.171313 (+ 571us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 12:03:13.185148 (+ 13835us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 12:03:13.185394 (+ 246us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36318: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":69}
W20260811 12:03:13.185787 19375 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36318: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260811 12:03:13.186029 19420 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:13.165681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34115 (local address 127.0.0.1:36318)
0811 12:03:13.166321 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:13.166341 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:13.166509 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:13.166914 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:13.166920 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:13.166935 (+ 15us) client_negotiation.cc:189] Negotiated authn=JWT
0811 12:03:13.167261 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:13.167273 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:13.168329 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:13.168336 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:13.170088 (+ 1752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:13.170104 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:13.170847 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:13.170854 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:13.171043 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:13.171089 (+ 46us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 12:03:13.185358 (+ 14269us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 12:03:13.185631 (+ 273us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.221.190:34115: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":325,"threads_started":1}
I20260811 12:03:13.187414 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 19259
I20260811 12:03:13.188556 19363 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:03:13.283488 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 19259
2026-08-11T12:03:13Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (729 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (417 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (417 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:03:14 dist-test-slave-mjjr krb5kdc[19440](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:03:14 dist-test-slave-mjjr krb5kdc[19440](info): set up 2 sockets
Aug 11 12:03:14 dist-test-slave-mjjr krb5kdc[19440](info): commencing operation
krb5kdc: starting...
W20260811 12:03:15.412518 10102 mini_kdc.cc:121] Time spent starting KDC: real 1.250s user 0.006s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Aug 11 12:03:15 dist-test-slave-mjjr krb5kdc[19440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:03:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:03:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.221.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
I20260811 12:03:15.640805 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:46233
--webserver_interface=127.9.221.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:36705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:46233
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:03:15.966132 19456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:03:15.966400 19456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:03:15.966562 19456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:03:15.973629 19456 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:03:15.973724 19456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:03:15.973778 19456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:03:15.973879 19456 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:03:15.973970 19456 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:03:15.983487 19456 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:36705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:46233
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:46233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.221.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:03:15.985188 19456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:03:15.986521 19456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:03:15.998124 19468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:15.998203 19467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:15.998193 19470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:03:15.998976 19456 server_base.cc:1034] running on GCE node
I20260811 12:03:15.999799 19456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:03:16.001132 19456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:03:16.002365 19456 hybrid_clock.cc:648] HybridClock initialized: now 1786449796002328 us; error 46 us; skew 500 ppm
I20260811 12:03:16.002696 19456 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:03:16 dist-test-slave-mjjr krb5kdc[19440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449796, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:03:16.010219 19456 init.cc:377] Logged in from keytab as kudu/127.9.221.190@KRBTEST.COM (short username kudu)
I20260811 12:03:16.011826 19456 webserver.cc:460] Webserver started at http://127.9.221.190:34705/ using document root <none> and password file <none>
I20260811 12:03:16.012503 19456 fs_manager.cc:359] Metadata directory not provided
I20260811 12:03:16.012661 19456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:03:16.012987 19456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:03:16.016079 19456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "ad5bbfed756d42738efe297cb7e245fa"
format_stamp: "Formatted at 2026-08-11 12:03:16 on dist-test-slave-mjjr"
server_key: "7d29fe3e4878f7ced4be7e682d5924d8"
server_key_iv: "421c67b84a61473bed84c7d0069b4e02"
server_key_version: "encryptionkey@0"
I20260811 12:03:16.016855 19456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "ad5bbfed756d42738efe297cb7e245fa"
format_stamp: "Formatted at 2026-08-11 12:03:16 on dist-test-slave-mjjr"
server_key: "7d29fe3e4878f7ced4be7e682d5924d8"
server_key_iv: "421c67b84a61473bed84c7d0069b4e02"
server_key_version: "encryptionkey@0"
I20260811 12:03:16.025421 19456 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.004s sys 0.004s
I20260811 12:03:16.028815 19483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:16.030103 19456 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:03:16.030277 19456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "ad5bbfed756d42738efe297cb7e245fa"
format_stamp: "Formatted at 2026-08-11 12:03:16 on dist-test-slave-mjjr"
server_key: "7d29fe3e4878f7ced4be7e682d5924d8"
server_key_iv: "421c67b84a61473bed84c7d0069b4e02"
server_key_version: "encryptionkey@0"
I20260811 12:03:16.030478 19456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:03:16.045529 19456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:03:16.049791 19456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:03:16.050087 19456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:03:16.068539 19456 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.190:46233
I20260811 12:03:16.068624 19580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.190:46233 every 8 connection(s)
I20260811 12:03:16.069900 19456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:03:16.072000 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 19456
I20260811 12:03:16.072247 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:03:16.073094 10102 external_mini_cluster.cc:1428] Setting key 5703d4146252dde4fe94544207730ef2
I20260811 12:03:16.074301 19581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 12:03:16 dist-test-slave-mjjr krb5kdc[19440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:03:16.089111 19581 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad5bbfed756d42738efe297cb7e245fa: Bootstrap starting.
I20260811 12:03:16.092403 19581 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad5bbfed756d42738efe297cb7e245fa: Neither blocks nor log segments found. Creating new log.
I20260811 12:03:16.093391 19581 log.cc:824] T 00000000000000000000000000000000 P ad5bbfed756d42738efe297cb7e245fa: Log is configured to *not* fsync() on all Append() calls
I20260811 12:03:16.096210 19585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:16.075159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59494 (local address 127.9.221.190:46233)
0811 12:03:16.075998 (+ 839us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:16.076009 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:16.076050 (+ 41us) server_negotiation.cc:407] Connection header received
0811 12:03:16.077634 (+ 1584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:16.077654 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:16.078015 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:16.078344 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:16.080239 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:16.081594 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:16.083343 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:16.084075 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:16.084197 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:16.088968 (+ 4771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:16.088976 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:16.088988 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:16.089022 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:16.092230 (+ 3208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:16.092956 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:16.092967 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:16.092972 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:16.093127 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:16.093505 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:16.093511 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:16.093514 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:16.094057 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:16.094414 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:16.094832 (+ 418us) server_negotiation.cc:299] Negotiation successful
0811 12:03:16.095243 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":380,"threads_started":1}
I20260811 12:03:16.096398 19581 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad5bbfed756d42738efe297cb7e245fa: No bootstrap required, opened a new log
I20260811 12:03:16.104339 19581 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad5bbfed756d42738efe297cb7e245fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad5bbfed756d42738efe297cb7e245fa" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 46233 } }
I20260811 12:03:16.104704 19581 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad5bbfed756d42738efe297cb7e245fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:03:16.104811 19581 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad5bbfed756d42738efe297cb7e245fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad5bbfed756d42738efe297cb7e245fa, State: Initialized, Role: FOLLOWER
I20260811 12:03:16.105342 19581 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad5bbfed756d42738efe297cb7e245fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad5bbfed756d42738efe297cb7e245fa" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 46233 } }
I20260811 12:03:16.105633 19581 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad5bbfed756d42738efe297cb7e245fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:03:16.105774 19581 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad5bbfed756d42738efe297cb7e245fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:03:16.105960 19581 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad5bbfed756d42738efe297cb7e245fa [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:03:16.108518 19581 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad5bbfed756d42738efe297cb7e245fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad5bbfed756d42738efe297cb7e245fa" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 46233 } }
I20260811 12:03:16.109018 19581 leader_election.cc:302] T 00000000000000000000000000000000 P ad5bbfed756d42738efe297cb7e245fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad5bbfed756d42738efe297cb7e245fa; no voters:
I20260811 12:03:16.109622 19581 leader_election.cc:288] T 00000000000000000000000000000000 P ad5bbfed756d42738efe297cb7e245fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:03:16.109717 19587 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad5bbfed756d42738efe297cb7e245fa [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:03:16.110498 19587 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad5bbfed756d42738efe297cb7e245fa [term 1 LEADER]: Becoming Leader. State: Replica: ad5bbfed756d42738efe297cb7e245fa, State: Running, Role: LEADER
I20260811 12:03:16.111002 19587 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad5bbfed756d42738efe297cb7e245fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad5bbfed756d42738efe297cb7e245fa" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 46233 } }
I20260811 12:03:16.111713 19581 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad5bbfed756d42738efe297cb7e245fa [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:03:16.113870 19588 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad5bbfed756d42738efe297cb7e245fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad5bbfed756d42738efe297cb7e245fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad5bbfed756d42738efe297cb7e245fa" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 46233 } } }
I20260811 12:03:16.114251 19588 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad5bbfed756d42738efe297cb7e245fa [sys.catalog]: This master's current role is: LEADER
I20260811 12:03:16.115576 19591 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad5bbfed756d42738efe297cb7e245fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad5bbfed756d42738efe297cb7e245fa. Latest consensus state: current_term: 1 leader_uuid: "ad5bbfed756d42738efe297cb7e245fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad5bbfed756d42738efe297cb7e245fa" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 46233 } } }
I20260811 12:03:16.115942 19591 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad5bbfed756d42738efe297cb7e245fa [sys.catalog]: This master's current role is: LEADER
I20260811 12:03:16.116328 19597 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:03:16.121467 19597 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:03:16.129737 19597 catalog_manager.cc:1347] Generated new cluster ID: 8c164d6960e34c5aacc24cef836f4fef
I20260811 12:03:16.129853 19597 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:03:16.139981 19597 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:03:16.141251 19597 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:03:16.148113 19597 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad5bbfed756d42738efe297cb7e245fa: Generated new TSK 0
I20260811 12:03:16.148910 19597 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.221.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab.
Entry for principal kudu/127.9.221.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab.
I20260811 12:03:16.211907 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.129:0
--local_ip_for_outbound_sockets=127.9.221.129
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:46233
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab
--principal=kudu/127.9.221.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:36705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:03:16.540274 19616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:03:16.540566 19616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:03:16.540723 19616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:03:16.547438 19616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:03:16.547529 19616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:03:16.547633 19616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.129
I20260811 12:03:16.556738 19616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:36705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab
--principal=kudu/127.9.221.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:46233
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.129
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:03:16.557807 19616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:03:16.559015 19616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:03:16.571781 19628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:16.571794 19624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:16.572037 19625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:03:16.572789 19616 server_base.cc:1034] running on GCE node
I20260811 12:03:16.573359 19616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:03:16.574280 19616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:03:16.575521 19616 hybrid_clock.cc:648] HybridClock initialized: now 1786449796575478 us; error 58 us; skew 500 ppm
I20260811 12:03:16.575856 19616 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:03:16 dist-test-slave-mjjr krb5kdc[19440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449796, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:03:16.578967 19616 init.cc:377] Logged in from keytab as kudu/127.9.221.129@KRBTEST.COM (short username kudu)
I20260811 12:03:16.580413 19616 webserver.cc:460] Webserver started at http://127.9.221.129:38359/ using document root <none> and password file <none>
I20260811 12:03:16.581094 19616 fs_manager.cc:359] Metadata directory not provided
I20260811 12:03:16.581250 19616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:03:16.581573 19616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:03:16.584491 19616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-0/data/instance:
uuid: "03d2e079911c4e7baecade654fbbac4d"
format_stamp: "Formatted at 2026-08-11 12:03:16 on dist-test-slave-mjjr"
server_key: "ad029fd443631a0602da1d9003d0f690"
server_key_iv: "62f168d9b4a0e0adbf9e6114569cb5a1"
server_key_version: "encryptionkey@0"
I20260811 12:03:16.585289 19616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance:
uuid: "03d2e079911c4e7baecade654fbbac4d"
format_stamp: "Formatted at 2026-08-11 12:03:16 on dist-test-slave-mjjr"
server_key: "ad029fd443631a0602da1d9003d0f690"
server_key_iv: "62f168d9b4a0e0adbf9e6114569cb5a1"
server_key_version: "encryptionkey@0"
I20260811 12:03:16.590342 19616 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:03:16.593827 19638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:16.595184 19616 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:03:16.595350 19616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-0/wal
uuid: "03d2e079911c4e7baecade654fbbac4d"
format_stamp: "Formatted at 2026-08-11 12:03:16 on dist-test-slave-mjjr"
server_key: "ad029fd443631a0602da1d9003d0f690"
server_key_iv: "62f168d9b4a0e0adbf9e6114569cb5a1"
server_key_version: "encryptionkey@0"
I20260811 12:03:16.595571 19616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:03:16.611363 19616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:03:16.615188 19616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:03:16.615612 19616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:03:16.616643 19616 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:03:16.618017 19616 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:03:16.618124 19616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:16.618278 19616 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:03:16.618352 19616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:16.652427 19616 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.129:36311
I20260811 12:03:16.652447 19813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.129:36311 every 8 connection(s)
I20260811 12:03:16.653753 19616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
I20260811 12:03:16.662853 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 19616
I20260811 12:03:16.663085 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance
I20260811 12:03:16.663923 10102 external_mini_cluster.cc:1428] Setting key 8728b5fe6949302c28f037ba29fadcba
Aug 11 12:03:16 dist-test-slave-mjjr krb5kdc[19440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449796, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:03:16.674831 19818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:16.656336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:43741 (local address 127.9.221.190:46233)
0811 12:03:16.656779 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:16.656784 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:16.657752 (+ 968us) server_negotiation.cc:407] Connection header received
0811 12:03:16.659566 (+ 1814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:16.659572 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:16.659733 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:16.659851 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:16.661367 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:16.662125 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:16.664629 (+ 2504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:16.665152 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:16.665256 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:16.669315 (+ 4059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:16.669320 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:16.669323 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:16.669353 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:16.671901 (+ 2548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:16.672603 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:16.672610 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:16.672614 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:16.672743 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:16.673159 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:16.673166 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:16.673169 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:16.673421 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:16.673580 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:16.674263 (+ 683us) server_negotiation.cc:299] Negotiation successful
0811 12:03:16.674481 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":220,"threads_started":1}
I20260811 12:03:16.675818 19819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:16.656882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:46233 (local address 127.9.221.129:43741)
0811 12:03:16.657588 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:16.657631 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:16.659286 (+ 1655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:16.660095 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:16.660102 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:16.660594 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:16.661162 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:16.661179 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:16.662362 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:16.662368 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:16.664455 (+ 2087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:16.664473 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:16.665424 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:16.665432 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:16.665722 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:16.666436 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:16.666482 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:16.669104 (+ 2622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:16.672052 (+ 2948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:16.672060 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:16.672075 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:16.672486 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:16.672856 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:16.672861 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:16.672865 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:16.673049 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:16.673550 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:16.673558 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:16.673895 (+ 337us) client_negotiation.cc:769] Sending connection context
0811 12:03:16.674153 (+ 258us) client_negotiation.cc:240] Negotiation successful
0811 12:03:16.674648 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":281,"threads_started":1}
I20260811 12:03:16.677299 19815 heartbeater.cc:347] Connected to a master server at 127.9.221.190:46233
I20260811 12:03:16.677671 19815 heartbeater.cc:464] Registering TS with master...
I20260811 12:03:16.678525 19815 heartbeater.cc:511] Master 127.9.221.190:46233 requested a full tablet report, sending...
I20260811 12:03:16.680982 19517 ts_manager.cc:194] Registered new tserver with Master: 03d2e079911c4e7baecade654fbbac4d (127.9.221.129:36311)
I20260811 12:03:16.683156 19517 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.129@KRBTEST.COM'} at 127.9.221.129:43741
WARNING: no policy specified for kudu/127.9.221.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab.
Entry for principal kudu/127.9.221.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab.
I20260811 12:03:16.716441 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.130:0
--local_ip_for_outbound_sockets=127.9.221.130
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:46233
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab
--principal=kudu/127.9.221.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:36705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:03:17.047816 19822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:03:17.048092 19822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:03:17.048187 19822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:03:17.055416 19822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:03:17.055539 19822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:03:17.055698 19822 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.130
I20260811 12:03:17.065478 19822 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:36705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab
--principal=kudu/127.9.221.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:46233
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.130
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:03:17.066777 19822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:03:17.068086 19822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:03:17.081529 19829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:17.081835 19830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:17.081576 19835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:03:17.082681 19822 server_base.cc:1034] running on GCE node
I20260811 12:03:17.083384 19822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:03:17.084519 19822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:03:17.085774 19822 hybrid_clock.cc:648] HybridClock initialized: now 1786449797085715 us; error 33 us; skew 500 ppm
I20260811 12:03:17.086138 19822 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:03:17 dist-test-slave-mjjr krb5kdc[19440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449797, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:03:17.092654 19822 init.cc:377] Logged in from keytab as kudu/127.9.221.130@KRBTEST.COM (short username kudu)
I20260811 12:03:17.094260 19822 webserver.cc:460] Webserver started at http://127.9.221.130:45309/ using document root <none> and password file <none>
I20260811 12:03:17.094988 19822 fs_manager.cc:359] Metadata directory not provided
I20260811 12:03:17.095144 19822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:03:17.095464 19822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:03:17.098452 19822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-1/data/instance:
uuid: "1276b41640354514b3993f276c181c49"
format_stamp: "Formatted at 2026-08-11 12:03:17 on dist-test-slave-mjjr"
server_key: "9d78bb05ee64d854227abf043064f827"
server_key_iv: "aa1ec0a9369599cffef5e2d0dc547daf"
server_key_version: "encryptionkey@0"
I20260811 12:03:17.099287 19822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance:
uuid: "1276b41640354514b3993f276c181c49"
format_stamp: "Formatted at 2026-08-11 12:03:17 on dist-test-slave-mjjr"
server_key: "9d78bb05ee64d854227abf043064f827"
server_key_iv: "aa1ec0a9369599cffef5e2d0dc547daf"
server_key_version: "encryptionkey@0"
I20260811 12:03:17.104494 19822 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260811 12:03:17.107965 19845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:17.109257 19822 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 12:03:17.109441 19822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-1/wal
uuid: "1276b41640354514b3993f276c181c49"
format_stamp: "Formatted at 2026-08-11 12:03:17 on dist-test-slave-mjjr"
server_key: "9d78bb05ee64d854227abf043064f827"
server_key_iv: "aa1ec0a9369599cffef5e2d0dc547daf"
server_key_version: "encryptionkey@0"
I20260811 12:03:17.109652 19822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:03:17.150336 19822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:03:17.154429 19822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:03:17.154747 19822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:03:17.155841 19822 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:03:17.157220 19822 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:03:17.157330 19822 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:17.157481 19822 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:03:17.157558 19822 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:17.191229 19822 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.130:43177
I20260811 12:03:17.191288 20017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.130:43177 every 8 connection(s)
I20260811 12:03:17.192595 19822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
I20260811 12:03:17.199709 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 19822
I20260811 12:03:17.199931 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance
I20260811 12:03:17.200371 10102 external_mini_cluster.cc:1428] Setting key b752912fc44ef27e0850952e1a4ed20d
Aug 11 12:03:17 dist-test-slave-mjjr krb5kdc[19440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449797, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.130@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:03:17.215873 20027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:17.195256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:51935 (local address 127.9.221.190:46233)
0811 12:03:17.195778 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:17.195783 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:17.196622 (+ 839us) server_negotiation.cc:407] Connection header received
0811 12:03:17.198429 (+ 1807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:17.198435 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:17.198539 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:17.198662 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:17.200495 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:17.201464 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:17.203554 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:17.204064 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:17.204166 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:17.208651 (+ 4485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:17.208660 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:17.208668 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:17.208713 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:17.212245 (+ 3532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:17.213214 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:17.213222 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:17.213228 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:17.213392 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:17.213813 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:17.213821 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:17.213847 (+ 26us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:17.214230 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:17.214395 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:17.215070 (+ 675us) server_negotiation.cc:299] Negotiation successful
0811 12:03:17.215324 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":260,"threads_started":1}
I20260811 12:03:17.216523 20028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:17.195756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:46233 (local address 127.9.221.130:51935)
0811 12:03:17.196447 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:17.196492 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:17.198153 (+ 1661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:17.198902 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:17.198910 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:17.199506 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:17.200321 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:17.200342 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:17.201621 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:17.201626 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:17.203400 (+ 1774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:17.203417 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:17.204294 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:17.204303 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:17.204625 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:17.205364 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:17.205396 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:17.208410 (+ 3014us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:17.212391 (+ 3981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:17.212402 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:17.212422 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:17.213071 (+ 649us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:17.213512 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:17.213516 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:17.213518 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:17.213694 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:17.214379 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:17.214384 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:17.214689 (+ 305us) client_negotiation.cc:769] Sending connection context
0811 12:03:17.214921 (+ 232us) client_negotiation.cc:240] Negotiation successful
0811 12:03:17.215311 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":288,"threads_started":1}
I20260811 12:03:17.218142 20019 heartbeater.cc:347] Connected to a master server at 127.9.221.190:46233
I20260811 12:03:17.218475 20019 heartbeater.cc:464] Registering TS with master...
I20260811 12:03:17.219400 20019 heartbeater.cc:511] Master 127.9.221.190:46233 requested a full tablet report, sending...
I20260811 12:03:17.220988 19517 ts_manager.cc:194] Registered new tserver with Master: 1276b41640354514b3993f276c181c49 (127.9.221.130:43177)
I20260811 12:03:17.221820 19517 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.130@KRBTEST.COM'} at 127.9.221.130:51935
WARNING: no policy specified for kudu/127.9.221.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab.
Entry for principal kudu/127.9.221.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab.
I20260811 12:03:17.255435 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.131:0
--local_ip_for_outbound_sockets=127.9.221.131
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:46233
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab
--principal=kudu/127.9.221.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:36705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:03:17.593730 20034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:03:17.594002 20034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:03:17.594168 20034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:03:17.601626 20034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:03:17.601754 20034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:03:17.601949 20034 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.131
I20260811 12:03:17.611783 20034 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:36705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab
--principal=kudu/127.9.221.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:46233
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.131
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:03:17.613250 20034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:03:17.614794 20034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:03:17.627885 20045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:17.627945 20041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:17.627849 20042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:03:17.628573 20034 server_base.cc:1034] running on GCE node
I20260811 12:03:17.629194 20034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:03:17.630095 20034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:03:17.631320 20034 hybrid_clock.cc:648] HybridClock initialized: now 1786449797631271 us; error 45 us; skew 500 ppm
I20260811 12:03:17.631630 20034 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:03:17 dist-test-slave-mjjr krb5kdc[19440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449797, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:03:17.634784 20034 init.cc:377] Logged in from keytab as kudu/127.9.221.131@KRBTEST.COM (short username kudu)
I20260811 12:03:17.636201 20034 webserver.cc:460] Webserver started at http://127.9.221.131:40879/ using document root <none> and password file <none>
I20260811 12:03:17.636842 20034 fs_manager.cc:359] Metadata directory not provided
I20260811 12:03:17.636988 20034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:03:17.637293 20034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:03:17.640169 20034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-2/data/instance:
uuid: "1dee717675ca42ddb1507035b4a2780f"
format_stamp: "Formatted at 2026-08-11 12:03:17 on dist-test-slave-mjjr"
server_key: "97e3541bbd25edc5d8b0da7e6b56ab29"
server_key_iv: "c686992e31fedbe643eeacb7ae118876"
server_key_version: "encryptionkey@0"
I20260811 12:03:17.640925 20034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance:
uuid: "1dee717675ca42ddb1507035b4a2780f"
format_stamp: "Formatted at 2026-08-11 12:03:17 on dist-test-slave-mjjr"
server_key: "97e3541bbd25edc5d8b0da7e6b56ab29"
server_key_iv: "c686992e31fedbe643eeacb7ae118876"
server_key_version: "encryptionkey@0"
I20260811 12:03:17.645830 20034 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:03:17.649070 20057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:17.650298 20034 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 12:03:17.650481 20034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-2/wal
uuid: "1dee717675ca42ddb1507035b4a2780f"
format_stamp: "Formatted at 2026-08-11 12:03:17 on dist-test-slave-mjjr"
server_key: "97e3541bbd25edc5d8b0da7e6b56ab29"
server_key_iv: "c686992e31fedbe643eeacb7ae118876"
server_key_version: "encryptionkey@0"
I20260811 12:03:17.650678 20034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:03:17.677380 20034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:03:17.681223 20034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:03:17.681527 20034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:03:17.682469 20034 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:03:17.683785 20034 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:03:17.683888 20034 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:17.684046 20034 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:03:17.684118 20034 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:17.687002 19815 heartbeater.cc:503] Master 127.9.221.190:46233 was elected leader, sending a full tablet report...
I20260811 12:03:17.718243 20034 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.131:34343
I20260811 12:03:17.718333 20220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.131:34343 every 8 connection(s)
I20260811 12:03:17.719514 20034 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
I20260811 12:03:17.728034 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 20034
I20260811 12:03:17.728277 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance
I20260811 12:03:17.728693 10102 external_mini_cluster.cc:1428] Setting key bdc97e31970fc7eff29af054417c8103
Aug 11 12:03:17 dist-test-slave-mjjr krb5kdc[19440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449797, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.131@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:03:17.738740 20223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:17.721964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:59827 (local address 127.9.221.190:46233)
0811 12:03:17.722412 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:17.722417 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:17.723228 (+ 811us) server_negotiation.cc:407] Connection header received
0811 12:03:17.725111 (+ 1883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:17.725116 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:17.725212 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:17.725331 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:17.726860 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:17.727715 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:17.729498 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:17.730069 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:17.730213 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:17.733685 (+ 3472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:17.733690 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:17.733693 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:17.733732 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:17.736293 (+ 2561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:17.736946 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:17.736950 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:17.736952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:17.737060 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:17.737405 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:17.737408 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:17.737410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:17.737640 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:17.737753 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:17.738336 (+ 583us) server_negotiation.cc:299] Negotiation successful
0811 12:03:17.738485 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":238,"threads_started":1}
I20260811 12:03:17.739732 20225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:17.722370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:46233 (local address 127.9.221.131:59827)
0811 12:03:17.723074 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:17.723111 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:17.724823 (+ 1712us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:17.725575 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:17.725583 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:17.726117 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:17.726689 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:17.726706 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:17.727856 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:17.727865 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:17.729318 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:17.729333 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:17.730279 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:17.730284 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:17.730498 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:17.731206 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:17.731227 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:17.733448 (+ 2221us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:17.736421 (+ 2973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:17.736427 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:17.736437 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:17.736844 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:17.737151 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:17.737153 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:17.737156 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:17.737317 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:17.737745 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:17.737751 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:17.738038 (+ 287us) client_negotiation.cc:769] Sending connection context
0811 12:03:17.738253 (+ 215us) client_negotiation.cc:240] Negotiation successful
0811 12:03:17.738591 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":282,"threads_started":1}
I20260811 12:03:17.740988 20221 heartbeater.cc:347] Connected to a master server at 127.9.221.190:46233
I20260811 12:03:17.741242 20221 heartbeater.cc:464] Registering TS with master...
I20260811 12:03:17.741858 20221 heartbeater.cc:511] Master 127.9.221.190:46233 requested a full tablet report, sending...
I20260811 12:03:17.743530 19517 ts_manager.cc:194] Registered new tserver with Master: 1dee717675ca42ddb1507035b4a2780f (127.9.221.131:34343)
I20260811 12:03:17.744372 10102 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:03:17.744542 19517 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.131@KRBTEST.COM'} at 127.9.221.131:59827
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/krb5kdc/test-admin.keytab.
Aug 11 12:03:17 dist-test-slave-mjjr krb5kdc[19440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:03:17.774820 10102 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260811 12:03:17.779944 20234 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:17.777503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43918 (local address 127.9.221.129:36311)
0811 12:03:17.778098 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:17.778104 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:17.778132 (+ 28us) server_negotiation.cc:407] Connection header received
0811 12:03:17.778194 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:17.778201 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:17.778393 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:17.778535 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:17.779409 (+ 874us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43918: BlockingRecv error: recv got EOF from 127.0.0.1:43918 (error 108)
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":299,"threads_started":1}
W20260811 12:03:17.780085 10102 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/krb5kdc/krb5cc' of type FILE
Aug 11 12:03:17 dist-test-slave-mjjr krb5kdc[19440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:03:17.781512 10102 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 12:03:17 dist-test-slave-mjjr krb5kdc[19440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:03:17.782944 10102 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 12:03:17 dist-test-slave-mjjr krb5kdc[19440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.129@KRBTEST.COM
I20260811 12:03:17.798285 20234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:17.784618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43920 (local address 127.9.221.129:36311)
0811 12:03:17.784811 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:17.784817 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:17.785012 (+ 195us) server_negotiation.cc:407] Connection header received
0811 12:03:17.785204 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:17.785210 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:17.785272 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:17.785371 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:17.786365 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:17.787337 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:17.788648 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:17.789221 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:17.789363 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:17.793035 (+ 3672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:17.793046 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:17.793061 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:17.793100 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:17.795814 (+ 2714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:17.796524 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:17.796531 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:17.796534 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:17.796652 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:17.796980 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:17.796983 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:17.796985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:17.797332 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:17.797551 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:17.797827 (+ 276us) server_negotiation.cc:299] Negotiation successful
0811 12:03:17.798036 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260811 12:03:17.804220 20234 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:17.802380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43926 (local address 127.9.221.129:36311)
0811 12:03:17.802629 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:17.802634 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:17.802819 (+ 185us) server_negotiation.cc:407] Connection header received
0811 12:03:17.802932 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:17.802936 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:17.803015 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:17.803093 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:17.803993 (+ 900us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43926: BlockingRecv error: recv got EOF from 127.0.0.1:43926 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
W20260811 12:03:17.804502 10102 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/krb5kdc/krb5cc' of type FILE
Aug 11 12:03:17 dist-test-slave-mjjr krb5kdc[19440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:03:17.805951 10102 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 12:03:17 dist-test-slave-mjjr krb5kdc[19440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:03:17.807253 10102 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 12:03:17 dist-test-slave-mjjr krb5kdc[19440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.129@KRBTEST.COM
I20260811 12:03:17.820837 20234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:17.808603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43942 (local address 127.9.221.129:36311)
0811 12:03:17.808781 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:17.808787 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:17.808989 (+ 202us) server_negotiation.cc:407] Connection header received
0811 12:03:17.809090 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:17.809094 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:17.809145 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:17.809220 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:17.810040 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:17.810708 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:17.811917 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:17.812453 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:17.812554 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:17.815894 (+ 3340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:17.815899 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:17.815902 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:17.815932 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:17.818649 (+ 2717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:17.819212 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:17.819215 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:17.819218 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:17.819333 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:17.819680 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:17.819687 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:17.819689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:17.819940 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:17.820142 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:17.820340 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 12:03:17.820508 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260811 12:03:17.825098 20234 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:17.823531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43958 (local address 127.9.221.129:36311)
0811 12:03:17.823723 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:17.823727 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:17.823984 (+ 257us) server_negotiation.cc:407] Connection header received
0811 12:03:17.824105 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:17.824108 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:17.824170 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:17.824245 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:17.824859 (+ 614us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43958: BlockingRecv error: recv got EOF from 127.0.0.1:43958 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260811 12:03:17.825380 10102 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449743084027-10102-0/krb5kdc/krb5cc' of type FILE
Aug 11 12:03:17 dist-test-slave-mjjr krb5kdc[19440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:03:17.826876 10102 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 12:03:17 dist-test-slave-mjjr krb5kdc[19440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:03:17.828195 10102 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 12:03:17 dist-test-slave-mjjr krb5kdc[19440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.129@KRBTEST.COM
I20260811 12:03:17.842460 20234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:17.829407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43962 (local address 127.9.221.129:36311)
0811 12:03:17.829627 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:17.829632 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:17.829893 (+ 261us) server_negotiation.cc:407] Connection header received
0811 12:03:17.830084 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:17.830094 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:17.830147 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:17.830248 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:17.831184 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:17.831970 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:17.833185 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:17.833609 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:17.833747 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:17.837409 (+ 3662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:17.837414 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:17.837417 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:17.837447 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:17.839886 (+ 2439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:17.840513 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:17.840519 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:17.840523 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:17.840679 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:17.841063 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:17.841068 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:17.841073 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:17.841355 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:17.841685 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:17.841936 (+ 251us) server_negotiation.cc:299] Negotiation successful
0811 12:03:17.842146 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 12:03:17.844064 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 19616
I20260811 12:03:17.854933 20234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:17.844779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43976 (local address 127.9.221.129:36311)
0811 12:03:17.844924 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:17.844928 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:17.845216 (+ 288us) server_negotiation.cc:407] Connection header received
0811 12:03:17.845338 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:17.845342 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:17.845389 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:17.845472 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:17.846259 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:17.846923 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:17.848082 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:17.848534 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:17.848630 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:17.850447 (+ 1817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:17.850457 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:17.850460 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:17.850484 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:17.852792 (+ 2308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:17.853326 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:17.853329 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:17.853331 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:17.853445 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:17.853807 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:17.853810 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:17.853812 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:17.854041 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:17.854258 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:17.854514 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 12:03:17.854662 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 12:03:17.855481 19809 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:03:17.941478 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 19616
I20260811 12:03:17.964375 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 19822
Aug 11 12:03:17 dist-test-slave-mjjr krb5kdc[19440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.130@KRBTEST.COM
I20260811 12:03:17.979003 20253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:17.965225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56256 (local address 127.9.221.130:43177)
0811 12:03:17.965792 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:17.965799 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:17.965826 (+ 27us) server_negotiation.cc:407] Connection header received
0811 12:03:17.965889 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:17.965896 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:17.966105 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:17.966230 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:17.967212 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:17.968314 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:17.969467 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:17.969986 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:17.970077 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:17.973993 (+ 3916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:17.974003 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:17.974016 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:17.974045 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:17.976683 (+ 2638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:17.977285 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:17.977292 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:17.977294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:17.977405 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:17.977721 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:17.977724 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:17.977726 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:17.978022 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:17.978224 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:17.978502 (+ 278us) server_negotiation.cc:299] Negotiation successful
0811 12:03:17.978690 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":307,"threads_started":1}
I20260811 12:03:17.980320 20011 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:03:18.062561 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 19822
I20260811 12:03:18.087245 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 20034
Aug 11 12:03:18 dist-test-slave-mjjr krb5kdc[19440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.131@KRBTEST.COM
I20260811 12:03:18.102543 20256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:18.088284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41600 (local address 127.9.221.131:34343)
0811 12:03:18.089011 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:18.089018 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:18.089048 (+ 30us) server_negotiation.cc:407] Connection header received
0811 12:03:18.089107 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:18.089114 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:18.089322 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:18.089450 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:18.090444 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:18.091468 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:18.092898 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:18.093434 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:18.093535 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:18.097396 (+ 3861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:18.097407 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:18.097421 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:18.097451 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:18.100160 (+ 2709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:18.100781 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:18.100787 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:18.100789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:18.100900 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:18.101226 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:18.101229 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:18.101231 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:18.101541 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:18.101777 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:18.101978 (+ 201us) server_negotiation.cc:299] Negotiation successful
0811 12:03:18.102245 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":413,"threads_started":1}
I20260811 12:03:18.103865 20212 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:03:18.186792 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 20034
I20260811 12:03:18.210021 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 19456
I20260811 12:03:18.211217 19574 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:03:18.315191 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 19456
2026-08-11T12:03:18Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4202 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:03:18 dist-test-slave-mjjr krb5kdc[20261](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:03:18 dist-test-slave-mjjr krb5kdc[20261](info): set up 2 sockets
Aug 11 12:03:18 dist-test-slave-mjjr krb5kdc[20261](info): commencing operation
krb5kdc: starting...
W20260811 12:03:19.629628 10102 mini_kdc.cc:121] Time spent starting KDC: real 1.265s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Aug 11 12:03:19 dist-test-slave-mjjr krb5kdc[20261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:03:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:03:19Z Disabled control of system clock
WARNING: no policy specified for oryx/127.9.221.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.221.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.221.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.190.keytab.
Entry for principal oryx/127.9.221.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.190.keytab.
I20260811 12:03:19.860788 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:38537
--webserver_interface=127.9.221.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.190.keytab
--principal=oryx/127.9.221.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:37859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:38537
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:03:20.190963 20281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:03:20.191306 20281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:03:20.191407 20281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:03:20.198369 20281 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:03:20.198468 20281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:03:20.198513 20281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:03:20.198572 20281 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:03:20.198616 20281 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:03:20.208036 20281 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:37859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:38537
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.190.keytab
--principal=oryx/127.9.221.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:38537
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.221.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:03:20.209219 20281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:03:20.210441 20281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:03:20.222381 20287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:20.222473 20290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:20.222338 20292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:03:20.223721 20281 server_base.cc:1034] running on GCE node
I20260811 12:03:20.224601 20281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:03:20.225914 20281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:03:20.227123 20281 hybrid_clock.cc:648] HybridClock initialized: now 1786449800227073 us; error 45 us; skew 500 ppm
I20260811 12:03:20.227437 20281 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:03:20 dist-test-slave-mjjr krb5kdc[20261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449800, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.221.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:03:20.230592 20281 init.cc:377] Logged in from keytab as oryx/127.9.221.190@KRBTEST.COM (short username oryx)
I20260811 12:03:20.231945 20281 webserver.cc:460] Webserver started at http://127.9.221.190:37311/ using document root <none> and password file <none>
I20260811 12:03:20.232573 20281 fs_manager.cc:359] Metadata directory not provided
I20260811 12:03:20.232671 20281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:03:20.232899 20281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:03:20.235790 20281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "5b972c76ffa8496faf82371b3f1523f7"
format_stamp: "Formatted at 2026-08-11 12:03:20 on dist-test-slave-mjjr"
server_key: "96a9ac5573b4c4a9f239758893412a88"
server_key_iv: "07b7b914fe7b2f373dd73a0d48f7963f"
server_key_version: "encryptionkey@0"
I20260811 12:03:20.236493 20281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "5b972c76ffa8496faf82371b3f1523f7"
format_stamp: "Formatted at 2026-08-11 12:03:20 on dist-test-slave-mjjr"
server_key: "96a9ac5573b4c4a9f239758893412a88"
server_key_iv: "07b7b914fe7b2f373dd73a0d48f7963f"
server_key_version: "encryptionkey@0"
I20260811 12:03:20.241267 20281 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260811 12:03:20.244472 20302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:20.245687 20281 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:03:20.245864 20281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "5b972c76ffa8496faf82371b3f1523f7"
format_stamp: "Formatted at 2026-08-11 12:03:20 on dist-test-slave-mjjr"
server_key: "96a9ac5573b4c4a9f239758893412a88"
server_key_iv: "07b7b914fe7b2f373dd73a0d48f7963f"
server_key_version: "encryptionkey@0"
I20260811 12:03:20.246063 20281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:03:20.260397 20281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:03:20.264139 20281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:03:20.264407 20281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:03:20.282297 20281 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.190:38537
I20260811 12:03:20.282331 20382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.190:38537 every 8 connection(s)
I20260811 12:03:20.283793 20281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:03:20.287603 20383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:03:20.293308 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 20281
I20260811 12:03:20.293552 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:03:20.294371 10102 external_mini_cluster.cc:1428] Setting key bc83867f599eee83d8135fa2b96b00a2
I20260811 12:03:20.299733 20383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7: Bootstrap starting.
I20260811 12:03:20.303529 20383 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7: Neither blocks nor log segments found. Creating new log.
I20260811 12:03:20.304837 20383 log.cc:824] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7: Log is configured to *not* fsync() on all Append() calls
I20260811 12:03:20.308411 20383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7: No bootstrap required, opened a new log
Aug 11 12:03:20 dist-test-slave-mjjr krb5kdc[20261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM
I20260811 12:03:20.316294 20383 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } }
I20260811 12:03:20.316663 20383 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:03:20.316787 20383 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b972c76ffa8496faf82371b3f1523f7, State: Initialized, Role: FOLLOWER
I20260811 12:03:20.317548 20383 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } }
I20260811 12:03:20.317871 20383 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:03:20.318089 20383 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:03:20.318310 20383 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:03:20.318632 20389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:20.296548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56310 (local address 127.9.221.190:38537)
0811 12:03:20.297458 (+ 910us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:20.297473 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:20.297514 (+ 41us) server_negotiation.cc:407] Connection header received
0811 12:03:20.299242 (+ 1728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:20.299263 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:20.299614 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:20.300063 (+ 449us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:20.301957 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:20.303225 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:20.305098 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:20.306162 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:20.306327 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:20.310894 (+ 4567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:20.310901 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:20.310916 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:20.310963 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:20.314090 (+ 3127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:20.314810 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:20.314819 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:20.314825 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:20.315006 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:20.315401 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:20.315405 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:20.315407 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:20.315982 (+ 575us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:20.316301 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:20.316669 (+ 368us) server_negotiation.cc:299] Negotiation successful
0811 12:03:20.317087 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":693,"thread_start_us":447,"threads_started":1}
I20260811 12:03:20.320919 20383 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } }
I20260811 12:03:20.321481 20383 leader_election.cc:302] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b972c76ffa8496faf82371b3f1523f7; no voters:
I20260811 12:03:20.322084 20383 leader_election.cc:288] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:03:20.322201 20393 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:03:20.322923 20393 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 [term 1 LEADER]: Becoming Leader. State: Replica: 5b972c76ffa8496faf82371b3f1523f7, State: Running, Role: LEADER
I20260811 12:03:20.323378 20393 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } }
I20260811 12:03:20.324162 20383 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:03:20.326263 20396 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b972c76ffa8496faf82371b3f1523f7. Latest consensus state: current_term: 1 leader_uuid: "5b972c76ffa8496faf82371b3f1523f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } }
I20260811 12:03:20.326725 20396 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 [sys.catalog]: This master's current role is: LEADER
I20260811 12:03:20.327848 20394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b972c76ffa8496faf82371b3f1523f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } }
I20260811 12:03:20.328285 20394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 [sys.catalog]: This master's current role is: LEADER
I20260811 12:03:20.333061 20404 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:03:20.338622 20404 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:03:20.347466 20404 catalog_manager.cc:1347] Generated new cluster ID: 2bf09098d4e5453ea97c6c9ed333b76a
I20260811 12:03:20.347756 20404 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:03:20.357431 20404 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:03:20.359248 20404 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:03:20.374322 20404 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7: Generated new TSK 0
I20260811 12:03:20.375200 20404 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.9.221.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.221.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.221.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.129.keytab.
Entry for principal oryx/127.9.221.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.129.keytab.
I20260811 12:03:20.441051 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.129:0
--local_ip_for_outbound_sockets=127.9.221.129
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:38537
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.129.keytab
--principal=oryx/127.9.221.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:37859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:03:20.784096 20431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:03:20.784400 20431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:03:20.784495 20431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:03:20.791565 20431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:03:20.791723 20431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:03:20.791886 20431 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.129
I20260811 12:03:20.801426 20431 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:37859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.129.keytab
--principal=oryx/127.9.221.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:38537
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.129
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:03:20.802639 20431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:03:20.803946 20431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:03:20.817427 20438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:20.817400 20437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:20.818089 20441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:03:20.818333 20431 server_base.cc:1034] running on GCE node
I20260811 12:03:20.818985 20431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:03:20.820083 20431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:03:20.821280 20431 hybrid_clock.cc:648] HybridClock initialized: now 1786449800821223 us; error 49 us; skew 500 ppm
I20260811 12:03:20.821597 20431 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:03:20 dist-test-slave-mjjr krb5kdc[20261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449800, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.221.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:03:20.824839 20431 init.cc:377] Logged in from keytab as oryx/127.9.221.129@KRBTEST.COM (short username oryx)
I20260811 12:03:20.826803 20431 webserver.cc:460] Webserver started at http://127.9.221.129:44775/ using document root <none> and password file <none>
I20260811 12:03:20.827450 20431 fs_manager.cc:359] Metadata directory not provided
I20260811 12:03:20.827557 20431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:03:20.827977 20431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:03:20.830834 20431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-0/data/instance:
uuid: "5d3b7729ac6448e1a27b453a8f278fbc"
format_stamp: "Formatted at 2026-08-11 12:03:20 on dist-test-slave-mjjr"
server_key: "fe457c062d5f8448203fe144d6dc7e82"
server_key_iv: "60ec8dcdd1a777a36a82f9b2a18ecfd4"
server_key_version: "encryptionkey@0"
I20260811 12:03:20.831558 20431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance:
uuid: "5d3b7729ac6448e1a27b453a8f278fbc"
format_stamp: "Formatted at 2026-08-11 12:03:20 on dist-test-slave-mjjr"
server_key: "fe457c062d5f8448203fe144d6dc7e82"
server_key_iv: "60ec8dcdd1a777a36a82f9b2a18ecfd4"
server_key_version: "encryptionkey@0"
I20260811 12:03:20.836835 20431 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 12:03:20.840191 20449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:20.841423 20431 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:03:20.841568 20431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-0/wal
uuid: "5d3b7729ac6448e1a27b453a8f278fbc"
format_stamp: "Formatted at 2026-08-11 12:03:20 on dist-test-slave-mjjr"
server_key: "fe457c062d5f8448203fe144d6dc7e82"
server_key_iv: "60ec8dcdd1a777a36a82f9b2a18ecfd4"
server_key_version: "encryptionkey@0"
I20260811 12:03:20.841715 20431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:03:20.862345 20431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:03:20.866330 20431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:03:20.866592 20431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:03:20.867446 20431 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:03:20.868711 20431 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:03:20.868786 20431 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:20.868876 20431 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:03:20.868919 20431 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:20.902843 20431 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.129:34021
I20260811 12:03:20.902890 20608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.129:34021 every 8 connection(s)
I20260811 12:03:20.904175 20431 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
I20260811 12:03:20.906560 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 20431
I20260811 12:03:20.906838 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance
I20260811 12:03:20.907606 10102 external_mini_cluster.cc:1428] Setting key d46f562c0775ae620a15cb6efcf654a8
Aug 11 12:03:20 dist-test-slave-mjjr krb5kdc[20261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449800, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.221.129@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM
I20260811 12:03:20.928753 20611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:20.906825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:33477 (local address 127.9.221.190:38537)
0811 12:03:20.907352 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:20.907359 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:20.908529 (+ 1170us) server_negotiation.cc:407] Connection header received
0811 12:03:20.910936 (+ 2407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:20.910940 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:20.911011 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:20.911098 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:20.913358 (+ 2260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:20.914087 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:20.916622 (+ 2535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:20.917006 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:20.917105 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:20.921598 (+ 4493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:20.921607 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:20.921615 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:20.921661 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:20.925099 (+ 3438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:20.926020 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:20.926029 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:20.926035 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:20.926197 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:20.926647 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:20.926654 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:20.926660 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:20.927048 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:20.927219 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:20.927924 (+ 705us) server_negotiation.cc:299] Negotiation successful
0811 12:03:20.928209 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":268,"threads_started":1}
I20260811 12:03:20.929379 20613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:20.907422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:38537 (local address 127.9.221.129:33477)
0811 12:03:20.908363 (+ 941us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:20.908403 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:20.910637 (+ 2234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:20.911386 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:20.911401 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:20.912274 (+ 873us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:20.913163 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:20.913194 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:20.914269 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:20.914277 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:20.916464 (+ 2187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:20.916484 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:20.917252 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:20.917261 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:20.917587 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:20.918324 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:20.918365 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:20.921321 (+ 2956us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:20.925276 (+ 3955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:20.925286 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:20.925305 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:20.925861 (+ 556us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:20.926320 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:20.926323 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:20.926326 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:20.926526 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:20.927174 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:20.927180 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:20.927482 (+ 302us) client_negotiation.cc:769] Sending connection context
0811 12:03:20.927769 (+ 287us) client_negotiation.cc:240] Negotiation successful
0811 12:03:20.928203 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":364,"threads_started":1}
I20260811 12:03:20.931013 20609 heartbeater.cc:347] Connected to a master server at 127.9.221.190:38537
I20260811 12:03:20.931329 20609 heartbeater.cc:464] Registering TS with master...
I20260811 12:03:20.932183 20609 heartbeater.cc:511] Master 127.9.221.190:38537 requested a full tablet report, sending...
I20260811 12:03:20.934252 20326 ts_manager.cc:194] Registered new tserver with Master: 5d3b7729ac6448e1a27b453a8f278fbc (127.9.221.129:34021)
WARNING: no policy specified for oryx/127.9.221.130@KRBTEST.COM; defaulting to no policy
I20260811 12:03:20.935856 20326 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.221.129@KRBTEST.COM'} at 127.9.221.129:33477
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.221.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.221.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.130.keytab.
Entry for principal oryx/127.9.221.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.130.keytab.
I20260811 12:03:20.965132 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.130:0
--local_ip_for_outbound_sockets=127.9.221.130
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:38537
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.130.keytab
--principal=oryx/127.9.221.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:37859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:03:21.294968 20617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:03:21.295229 20617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:03:21.295389 20617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:03:21.302734 20617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:03:21.302861 20617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:03:21.303058 20617 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.130
I20260811 12:03:21.312584 20617 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:37859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.130.keytab
--principal=oryx/127.9.221.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:38537
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.130
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:03:21.313781 20617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:03:21.315039 20617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:03:21.328547 20626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:21.328588 20624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:21.328788 20623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:03:21.329515 20617 server_base.cc:1034] running on GCE node
I20260811 12:03:21.330085 20617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:03:21.330960 20617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:03:21.332182 20617 hybrid_clock.cc:648] HybridClock initialized: now 1786449801332141 us; error 50 us; skew 500 ppm
I20260811 12:03:21.332484 20617 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:03:21 dist-test-slave-mjjr krb5kdc[20261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449801, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.221.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:03:21.340085 20617 init.cc:377] Logged in from keytab as oryx/127.9.221.130@KRBTEST.COM (short username oryx)
I20260811 12:03:21.341468 20617 webserver.cc:460] Webserver started at http://127.9.221.130:35777/ using document root <none> and password file <none>
I20260811 12:03:21.342133 20617 fs_manager.cc:359] Metadata directory not provided
I20260811 12:03:21.342269 20617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:03:21.342598 20617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:03:21.345739 20617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-1/data/instance:
uuid: "eae73ca664404c1e9ccc0414bc2daa03"
format_stamp: "Formatted at 2026-08-11 12:03:21 on dist-test-slave-mjjr"
server_key: "9e5424694b4035e367dc4491bbcb8feb"
server_key_iv: "5d979aed036cd22e1a959155559b3862"
server_key_version: "encryptionkey@0"
I20260811 12:03:21.346568 20617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance:
uuid: "eae73ca664404c1e9ccc0414bc2daa03"
format_stamp: "Formatted at 2026-08-11 12:03:21 on dist-test-slave-mjjr"
server_key: "9e5424694b4035e367dc4491bbcb8feb"
server_key_iv: "5d979aed036cd22e1a959155559b3862"
server_key_version: "encryptionkey@0"
I20260811 12:03:21.351635 20617 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260811 12:03:21.354979 20639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:21.356233 20617 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:03:21.356413 20617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-1/wal
uuid: "eae73ca664404c1e9ccc0414bc2daa03"
format_stamp: "Formatted at 2026-08-11 12:03:21 on dist-test-slave-mjjr"
server_key: "9e5424694b4035e367dc4491bbcb8feb"
server_key_iv: "5d979aed036cd22e1a959155559b3862"
server_key_version: "encryptionkey@0"
I20260811 12:03:21.356602 20617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:03:21.401458 20617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:03:21.405428 20617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:03:21.405750 20617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:03:21.406776 20617 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:03:21.408150 20617 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:03:21.408224 20617 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:21.408397 20617 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:03:21.408452 20617 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:21.442749 20617 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.130:38881
I20260811 12:03:21.442807 20812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.130:38881 every 8 connection(s)
I20260811 12:03:21.444276 20617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
I20260811 12:03:21.450501 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 20617
I20260811 12:03:21.450716 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance
I20260811 12:03:21.451153 10102 external_mini_cluster.cc:1428] Setting key b47e0e43616a1fc94df66ebb91e1a5c1
Aug 11 12:03:21 dist-test-slave-mjjr krb5kdc[20261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449801, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.221.130@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM
I20260811 12:03:21.464942 20817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:21.446950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:33487 (local address 127.9.221.190:38537)
0811 12:03:21.447382 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:21.447387 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:21.448451 (+ 1064us) server_negotiation.cc:407] Connection header received
0811 12:03:21.450340 (+ 1889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:21.450345 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:21.450424 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:21.450576 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:21.452360 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:21.453175 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:21.454641 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:21.455052 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:21.455186 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:21.458764 (+ 3578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:21.458769 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:21.458772 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:21.458802 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:21.461856 (+ 3054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:21.462749 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:21.462753 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:21.462755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:21.462872 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:21.463330 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:21.463333 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:21.463335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:21.463574 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:21.463746 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:21.464544 (+ 798us) server_negotiation.cc:299] Negotiation successful
0811 12:03:21.464695 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":238,"threads_started":1}
I20260811 12:03:21.466526 20818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:21.447516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:38537 (local address 127.9.221.130:33487)
0811 12:03:21.448303 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:21.448340 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:21.449988 (+ 1648us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:21.450869 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:21.450889 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:21.451441 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:21.452186 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:21.452207 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:21.453328 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:21.453333 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:21.454499 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:21.454510 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:21.455268 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:21.455276 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:21.455512 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:21.456227 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:21.456252 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:21.458518 (+ 2266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:21.462005 (+ 3487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:21.462015 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:21.462030 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:21.462635 (+ 605us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:21.462973 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:21.462980 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:21.462986 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:21.463223 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:21.463711 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:21.463721 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:21.464148 (+ 427us) client_negotiation.cc:769] Sending connection context
0811 12:03:21.464450 (+ 302us) client_negotiation.cc:240] Negotiation successful
0811 12:03:21.465018 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":304,"threads_started":1}
I20260811 12:03:21.467942 20813 heartbeater.cc:347] Connected to a master server at 127.9.221.190:38537
I20260811 12:03:21.468238 20813 heartbeater.cc:464] Registering TS with master...
I20260811 12:03:21.468899 20813 heartbeater.cc:511] Master 127.9.221.190:38537 requested a full tablet report, sending...
I20260811 12:03:21.470297 20326 ts_manager.cc:194] Registered new tserver with Master: eae73ca664404c1e9ccc0414bc2daa03 (127.9.221.130:38881)
I20260811 12:03:21.471083 20326 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.221.130@KRBTEST.COM'} at 127.9.221.130:33487
WARNING: no policy specified for oryx/127.9.221.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.221.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.221.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.131.keytab.
Entry for principal oryx/127.9.221.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.131.keytab.
I20260811 12:03:21.504172 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.131:0
--local_ip_for_outbound_sockets=127.9.221.131
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:38537
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.131.keytab
--principal=oryx/127.9.221.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:37859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:03:21.840190 20822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:03:21.840529 20822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:03:21.840699 20822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:03:21.848096 20822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:03:21.848239 20822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:03:21.848405 20822 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.131
I20260811 12:03:21.858357 20822 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:37859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.131.keytab
--principal=oryx/127.9.221.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:38537
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.131
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:03:21.859761 20822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:03:21.861181 20822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:03:21.877247 20829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:21.877213 20831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:21.877489 20834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:03:21.878155 20822 server_base.cc:1034] running on GCE node
I20260811 12:03:21.878809 20822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:03:21.879781 20822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:03:21.881007 20822 hybrid_clock.cc:648] HybridClock initialized: now 1786449801880963 us; error 54 us; skew 500 ppm
I20260811 12:03:21.881323 20822 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:03:21 dist-test-slave-mjjr krb5kdc[20261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449801, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.221.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:03:21.884498 20822 init.cc:377] Logged in from keytab as oryx/127.9.221.131@KRBTEST.COM (short username oryx)
I20260811 12:03:21.885814 20822 webserver.cc:460] Webserver started at http://127.9.221.131:43067/ using document root <none> and password file <none>
I20260811 12:03:21.886433 20822 fs_manager.cc:359] Metadata directory not provided
I20260811 12:03:21.886551 20822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:03:21.886797 20822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:03:21.889645 20822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-2/data/instance:
uuid: "3d4a8b75d0574794b67bb8e235e307b2"
format_stamp: "Formatted at 2026-08-11 12:03:21 on dist-test-slave-mjjr"
server_key: "6a9bf2ea925d8e1edf23528384958ad6"
server_key_iv: "8de5cd9a3a166de349d0a9b51a0fff00"
server_key_version: "encryptionkey@0"
I20260811 12:03:21.890358 20822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance:
uuid: "3d4a8b75d0574794b67bb8e235e307b2"
format_stamp: "Formatted at 2026-08-11 12:03:21 on dist-test-slave-mjjr"
server_key: "6a9bf2ea925d8e1edf23528384958ad6"
server_key_iv: "8de5cd9a3a166de349d0a9b51a0fff00"
server_key_version: "encryptionkey@0"
I20260811 12:03:21.895776 20822 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 12:03:21.899060 20846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:21.900409 20822 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:03:21.900578 20822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-2/wal
uuid: "3d4a8b75d0574794b67bb8e235e307b2"
format_stamp: "Formatted at 2026-08-11 12:03:21 on dist-test-slave-mjjr"
server_key: "6a9bf2ea925d8e1edf23528384958ad6"
server_key_iv: "8de5cd9a3a166de349d0a9b51a0fff00"
server_key_version: "encryptionkey@0"
I20260811 12:03:21.900753 20822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:03:21.915501 20822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:03:21.919205 20822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:03:21.919497 20822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:03:21.920481 20822 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:03:21.921777 20822 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:03:21.921909 20822 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:21.922091 20822 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:03:21.922149 20822 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:21.938827 20609 heartbeater.cc:503] Master 127.9.221.190:38537 was elected leader, sending a full tablet report...
I20260811 12:03:21.956396 20822 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.131:39233
I20260811 12:03:21.956444 21011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.131:39233 every 8 connection(s)
I20260811 12:03:21.957679 20822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
I20260811 12:03:21.965883 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 20822
I20260811 12:03:21.966109 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance
I20260811 12:03:21.966635 10102 external_mini_cluster.cc:1428] Setting key 40b1d8c0b877a434f50978a9aebfa0fc
Aug 11 12:03:21 dist-test-slave-mjjr krb5kdc[20261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449801, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.221.131@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM
I20260811 12:03:21.977105 20817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:21.960262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:55801 (local address 127.9.221.190:38537)
0811 12:03:21.960494 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:21.960500 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:21.961706 (+ 1206us) server_negotiation.cc:407] Connection header received
0811 12:03:21.963356 (+ 1650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:21.963360 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:21.963426 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:21.963555 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:21.964980 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:21.965913 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:21.967764 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:21.968159 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:21.968297 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:21.971620 (+ 3323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:21.971625 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:21.971628 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:21.971708 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:21.974059 (+ 2351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:21.974762 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:21.974769 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:21.974772 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:21.974897 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:21.975324 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:21.975329 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:21.975332 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:21.975610 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:21.975828 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:21.976511 (+ 683us) server_negotiation.cc:299] Negotiation successful
0811 12:03:21.976741 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 12:03:21.978098 21014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:21.960808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:38537 (local address 127.9.221.131:55801)
0811 12:03:21.961530 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:21.961568 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:21.963117 (+ 1549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:21.963762 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:21.963769 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:21.964252 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:21.964830 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:21.964846 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:21.966099 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:21.966103 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:21.967562 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:21.967585 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:21.968399 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:21.968406 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:21.968636 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:21.969305 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:21.969326 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:21.971410 (+ 2084us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:21.974214 (+ 2804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:21.974222 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:21.974237 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:21.974646 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:21.975016 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:21.975021 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:21.975025 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:21.975211 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:21.975787 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:21.975796 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:21.976136 (+ 340us) client_negotiation.cc:769] Sending connection context
0811 12:03:21.976401 (+ 265us) client_negotiation.cc:240] Negotiation successful
0811 12:03:21.976899 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":274,"threads_started":1}
I20260811 12:03:21.979491 21012 heartbeater.cc:347] Connected to a master server at 127.9.221.190:38537
I20260811 12:03:21.979811 21012 heartbeater.cc:464] Registering TS with master...
I20260811 12:03:21.980593 21012 heartbeater.cc:511] Master 127.9.221.190:38537 requested a full tablet report, sending...
I20260811 12:03:21.982013 20326 ts_manager.cc:194] Registered new tserver with Master: 3d4a8b75d0574794b67bb8e235e307b2 (127.9.221.131:39233)
I20260811 12:03:21.982815 20326 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.221.131@KRBTEST.COM'} at 127.9.221.131:55801
I20260811 12:03:21.993927 10102 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.9.221.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.221.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.221.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.189.keytab.
Entry for principal oryx/127.9.221.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.189.keytab.
I20260811 12:03:22.044159 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.189:37829
--webserver_interface=127.9.221.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.189.keytab
--principal=oryx/127.9.221.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:37859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:38537,127.9.221.189:37829
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:03:22.374826 21021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:03:22.375102 21021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:03:22.375192 21021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:03:22.382282 21021 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:03:22.382395 21021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:03:22.382442 21021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:03:22.382493 21021 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:03:22.382530 21021 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:03:22.392680 21021 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:37859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:38537,127.9.221.189:37829
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.189.keytab
--principal=oryx/127.9.221.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.189:37829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.221.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
W20260811 12:03:22.393046 21021 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.9.221.190:38537,127.9.221.189:37829'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 12:03:22.394061 21021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:03:22.395315 21021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:03:22.407135 21031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:22.407219 21032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:22.407160 21036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:03:22.408191 21021 server_base.cc:1034] running on GCE node
I20260811 12:03:22.408985 21021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:03:22.410384 21021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:03:22.411674 21021 hybrid_clock.cc:648] HybridClock initialized: now 1786449802411578 us; error 37 us; skew 500 ppm
I20260811 12:03:22.412031 21021 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:03:22 dist-test-slave-mjjr krb5kdc[20261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449802, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.221.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:03:22.415202 21021 init.cc:377] Logged in from keytab as oryx/127.9.221.189@KRBTEST.COM (short username oryx)
I20260811 12:03:22.416610 21021 webserver.cc:460] Webserver started at http://127.9.221.189:44063/ using document root <none> and password file <none>
I20260811 12:03:22.417265 21021 fs_manager.cc:359] Metadata directory not provided
I20260811 12:03:22.417353 21021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:03:22.417609 21021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:03:22.420591 21021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-1/data/instance:
uuid: "37c923a543d6418694238c016281ad2c"
format_stamp: "Formatted at 2026-08-11 12:03:22 on dist-test-slave-mjjr"
server_key: "bdac6b2233a779a1e322bec1b33c02ca"
server_key_iv: "659f7a5e6e416b28ee05313b8f4b0f83"
server_key_version: "encryptionkey@0"
I20260811 12:03:22.421375 21021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-1/wal/instance:
uuid: "37c923a543d6418694238c016281ad2c"
format_stamp: "Formatted at 2026-08-11 12:03:22 on dist-test-slave-mjjr"
server_key: "bdac6b2233a779a1e322bec1b33c02ca"
server_key_iv: "659f7a5e6e416b28ee05313b8f4b0f83"
server_key_version: "encryptionkey@0"
I20260811 12:03:22.426824 21021 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 12:03:22.430495 21050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:22.431823 21021 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 12:03:22.432026 21021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-1/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-1/wal
uuid: "37c923a543d6418694238c016281ad2c"
format_stamp: "Formatted at 2026-08-11 12:03:22 on dist-test-slave-mjjr"
server_key: "bdac6b2233a779a1e322bec1b33c02ca"
server_key_iv: "659f7a5e6e416b28ee05313b8f4b0f83"
server_key_version: "encryptionkey@0"
I20260811 12:03:22.432292 21021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:03:22.473591 20813 heartbeater.cc:503] Master 127.9.221.190:38537 was elected leader, sending a full tablet report...
I20260811 12:03:22.480757 21021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:03:22.484843 21021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:03:22.485117 21021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:03:22.504251 21021 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.189:37829
I20260811 12:03:22.504307 21124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.189:37829 every 8 connection(s)
I20260811 12:03:22.505832 21021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-1/data/info.pb
I20260811 12:03:22.509440 21125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:03:22.515308 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 21021
I20260811 12:03:22.515547 21125 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:03:22 dist-test-slave-mjjr krb5kdc[20261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449802, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.221.189@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM
Aug 11 12:03:22 dist-test-slave-mjjr krb5kdc[20261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.221.189@KRBTEST.COM
I20260811 12:03:22.537763 21131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:22.518344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56314 (local address 127.9.221.190:38537)
0811 12:03:22.518825 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:22.518834 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:22.519482 (+ 648us) server_negotiation.cc:407] Connection header received
0811 12:03:22.520616 (+ 1134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:22.520624 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:22.520711 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:22.520833 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:22.522476 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:22.523732 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:22.525812 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:22.526315 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:22.526443 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:22.530310 (+ 3867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:22.530315 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:22.530318 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:22.530349 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:22.534113 (+ 3764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:22.534895 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:22.534901 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:22.534907 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:22.535070 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:22.535551 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:22.535555 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:22.535557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:22.535920 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:22.536091 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:22.537009 (+ 918us) server_negotiation.cc:299] Negotiation successful
0811 12:03:22.537264 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":245,"threads_started":1}
I20260811 12:03:22.538609 21133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:22.518394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:38537 (local address 127.0.0.1:56314)
0811 12:03:22.519316 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:22.519365 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:22.520352 (+ 987us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:22.520914 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:22.520920 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:22.521745 (+ 825us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:22.522291 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:22.522315 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:22.523857 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:22.523861 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:22.525656 (+ 1795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:22.525670 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:22.526542 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:22.526548 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:22.526705 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:22.527406 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:22.527441 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:22.530115 (+ 2674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:22.534232 (+ 4117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:22.534241 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:22.534249 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:22.534767 (+ 518us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:22.535182 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:22.535185 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:22.535187 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:22.535422 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:22.536110 (+ 688us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:22.536116 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:22.536498 (+ 382us) client_negotiation.cc:769] Sending connection context
0811 12:03:22.536865 (+ 367us) client_negotiation.cc:240] Negotiation successful
0811 12:03:22.537361 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":672,"spinlock_wait_cycles":13056,"thread_start_us":459,"threads_started":1}
I20260811 12:03:22.538635 21130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:22.517613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36400 (local address 127.9.221.189:37829)
0811 12:03:22.518396 (+ 783us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:22.518405 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:22.518443 (+ 38us) server_negotiation.cc:407] Connection header received
0811 12:03:22.520338 (+ 1895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:22.520358 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:22.520673 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:22.520866 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:22.522648 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:22.524374 (+ 1726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:22.526119 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:22.526842 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:22.526939 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:22.531384 (+ 4445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:22.531396 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:22.531411 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:22.531450 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:22.535207 (+ 3757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:22.536036 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:22.536044 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:22.536047 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:22.536203 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:22.536670 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:22.536674 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:22.536677 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:22.537079 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:22.537397 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:22.537778 (+ 381us) server_negotiation.cc:299] Negotiation successful
0811 12:03:22.538329 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":361,"threads_started":1}
I20260811 12:03:22.540727 21125 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.221.189" port: 37829 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:03:22 dist-test-slave-mjjr krb5kdc[20261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449802, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.221.189@KRBTEST.COM for oryx/127.9.221.189@KRBTEST.COM
I20260811 12:03:22.552460 21133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:22.541565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:37829 (local address 127.0.0.1:36402)
0811 12:03:22.541808 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:22.541823 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:22.541906 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:22.542151 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:22.542154 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:22.542437 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:22.542573 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:22.542580 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:22.543469 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:22.543472 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:22.544379 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:22.544390 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:22.545034 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:22.545038 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:22.545159 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:22.545632 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:22.545647 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:22.547609 (+ 1962us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:22.550426 (+ 2817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:22.550430 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:22.550432 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:22.550841 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:22.551163 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:22.551166 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:22.551168 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:22.551287 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:22.551793 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:22.551796 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:22.551904 (+ 108us) client_negotiation.cc:769] Sending connection context
0811 12:03:22.552019 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 12:03:22.552215 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181}
I20260811 12:03:22.552582 21130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:22.541571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36402 (local address 127.9.221.189:37829)
0811 12:03:22.541732 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:22.541736 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:22.541889 (+ 153us) server_negotiation.cc:407] Connection header received
0811 12:03:22.541988 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:22.541992 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:22.542043 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:22.542114 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:22.542688 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:22.543344 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:22.544517 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:22.544906 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:22.545006 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:22.547828 (+ 2822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:22.547833 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:22.547836 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:22.547862 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:22.550302 (+ 2440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:22.550947 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:22.550951 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:22.550953 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:22.551069 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:22.551372 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:22.551375 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:22.551377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:22.551616 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:22.551792 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:22.552109 (+ 317us) server_negotiation.cc:299] Negotiation successful
0811 12:03:22.552306 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":52}
I20260811 12:03:22.557991 21125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c: Bootstrap starting.
I20260811 12:03:22.561107 21125 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c: Neither blocks nor log segments found. Creating new log.
I20260811 12:03:22.562112 21125 log.cc:824] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c: Log is configured to *not* fsync() on all Append() calls
I20260811 12:03:22.564766 21125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c: No bootstrap required, opened a new log
I20260811 12:03:22.571730 21125 raft_consensus.cc:348] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } peers { permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 37829 } }
I20260811 12:03:22.572082 21125 raft_consensus.cc:374] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:03:22.572193 21125 raft_consensus.cc:729] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37c923a543d6418694238c016281ad2c, State: Initialized, Role: FOLLOWER
I20260811 12:03:22.572805 21125 consensus_queue.cc:260] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } peers { permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 37829 } }
I20260811 12:03:22.573637 21135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } peers { permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 37829 } } }
I20260811 12:03:22.573922 21135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:03:22.574612 21125 sys_catalog.cc:564] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c [sys.catalog]: configured and running, proceeding with master startup.
W20260811 12:03:22.583536 21153 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 12:03:22.583709 21153 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 12:03:22.587076 21021 master_runner.cc:428] Detected that this master 37c923a543d6418694238c016281ad2c is joining an existing cluster
I20260811 12:03:22.587193 21021 master_runner.cc:432] Initiating AddMaster RPC to add 127.9.221.189:37829
I20260811 12:03:22.617465 21131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:22.591554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56316 (local address 127.9.221.190:38537)
0811 12:03:22.591769 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:22.591778 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:22.599535 (+ 7757us) server_negotiation.cc:407] Connection header received
0811 12:03:22.600749 (+ 1214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:22.600757 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:22.600889 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:22.600975 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:22.602580 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:22.603838 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:22.606327 (+ 2489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:22.607074 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:22.607194 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:22.609815 (+ 2621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:22.609823 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:22.609830 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:22.609899 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:22.612889 (+ 2990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:22.613922 (+ 1033us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:22.613930 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:22.613936 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:22.614108 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:22.614672 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:22.614681 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:22.614686 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:22.615126 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:22.615282 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:22.616399 (+ 1117us) server_negotiation.cc:299] Negotiation successful
0811 12:03:22.616686 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 12:03:22.619148 21160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:22.591399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:38537 (local address 127.0.0.1:56316)
0811 12:03:22.599379 (+ 7980us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:22.599409 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:22.600541 (+ 1132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:22.601221 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:22.601225 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:22.601948 (+ 723us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:22.602352 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:22.602365 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:22.604073 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:22.604079 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:22.606117 (+ 2038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:22.606136 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:22.607558 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:22.607565 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:22.607804 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:22.608590 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:22.608621 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:22.609591 (+ 970us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:22.613050 (+ 3459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:22.613059 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:22.613067 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:22.613752 (+ 685us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:22.614297 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:22.614303 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:22.614307 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:22.614535 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:22.615682 (+ 1147us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:22.615687 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:22.615945 (+ 258us) client_negotiation.cc:769] Sending connection context
0811 12:03:22.618122 (+ 2177us) client_negotiation.cc:240] Negotiation successful
0811 12:03:22.618532 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7767,"spinlock_wait_cycles":46336,"thread_start_us":8180,"threads_started":1}
I20260811 12:03:22.622434 21130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:22.591338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36406 (local address 127.9.221.189:37829)
0811 12:03:22.591553 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:22.591560 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:22.601737 (+ 10177us) server_negotiation.cc:407] Connection header received
0811 12:03:22.601938 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:22.601943 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:22.602028 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:22.602098 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:22.603478 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:22.604584 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:22.606768 (+ 2184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:22.607544 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:22.607707 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:22.614585 (+ 6878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:22.614593 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:22.614597 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:22.614638 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:22.618709 (+ 4071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:22.619662 (+ 953us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:22.619671 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:22.619678 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:22.619858 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:22.620344 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:22.620351 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:22.620357 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:22.620828 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:22.621012 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:22.621595 (+ 583us) server_negotiation.cc:299] Negotiation successful
0811 12:03:22.621893 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 12:03:22.625103 21161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:22.591973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:37829 (local address 127.0.0.1:36406)
0811 12:03:22.601645 (+ 9672us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:22.601668 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:22.601824 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:22.602290 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:22.602297 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:22.602984 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:22.603329 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:22.603342 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:22.604812 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:22.604817 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:22.606606 (+ 1789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:22.606624 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:22.607770 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:22.607779 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:22.607996 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:22.613167 (+ 5171us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:22.613195 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:22.614368 (+ 1173us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:22.618872 (+ 4504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:22.618878 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:22.618882 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:22.619490 (+ 608us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:22.620012 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:22.620017 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:22.620021 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:22.620200 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:22.621098 (+ 898us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:22.621106 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:22.621297 (+ 191us) client_negotiation.cc:769] Sending connection context
0811 12:03:22.622854 (+ 1557us) client_negotiation.cc:240] Negotiation successful
0811 12:03:22.624621 (+ 1767us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8438,"mutex_wait_us":3,"thread_start_us":8241,"threads_started":1}
WARNING: no policy specified for oryx/127.9.221.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.221.188@KRBTEST.COM" created.
Aug 11 12:03:22 dist-test-slave-mjjr krb5kdc[20261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449800, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.221.190@KRBTEST.COM for oryx/127.9.221.189@KRBTEST.COM
I20260811 12:03:22.646133 21130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:22.627035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36418 (local address 127.9.221.189:37829)
0811 12:03:22.627222 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:22.627228 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:22.628194 (+ 966us) server_negotiation.cc:407] Connection header received
0811 12:03:22.628572 (+ 378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:22.628579 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:22.628704 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:22.628813 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:22.630330 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:22.631418 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:22.633357 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:22.634027 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:22.634180 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:22.638316 (+ 4136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:22.638322 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:22.638328 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:22.638376 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:22.642237 (+ 3861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:22.643308 (+ 1071us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:22.643314 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:22.643318 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:22.643489 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:22.644064 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:22.644069 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:22.644073 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:22.644463 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:22.644615 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:22.645294 (+ 679us) server_negotiation.cc:299] Negotiation successful
0811 12:03:22.645536 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 12:03:22.646229 21166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:22.627173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:37829 (local address 127.0.0.1:36418)
0811 12:03:22.628058 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:22.628099 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:22.628320 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:22.628979 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:22.628991 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:22.629826 (+ 835us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:22.630133 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:22.630149 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:22.631612 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:22.631620 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:22.633188 (+ 1568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:22.633203 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:22.634216 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:22.634223 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:22.634437 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:22.635134 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:22.635160 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:22.638083 (+ 2923us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:22.642412 (+ 4329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:22.642431 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:22.642442 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:22.643136 (+ 694us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:22.643678 (+ 542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:22.643689 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:22.643696 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:22.643908 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:22.644624 (+ 716us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:22.644636 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:22.644885 (+ 249us) client_negotiation.cc:769] Sending connection context
0811 12:03:22.645172 (+ 287us) client_negotiation.cc:240] Negotiation successful
0811 12:03:22.645614 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":326,"threads_started":1}
I20260811 12:03:22.648952 20326 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.9.221.189:37829: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "37c923a543d6418694238c016281ad2c"
member_type: NON_VOTER
last_known_addr {
host: "127.9.221.189"
port: 37829
}
attrs {
promote: true
}
}
dest_uuid: "5b972c76ffa8496faf82371b3f1523f7"
cas_config_opid_index: -1
I20260811 12:03:22.650774 20326 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } peers { permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: NON_VOTER last_known_addr { host: "127.9.221.189" port: 37829 } attrs { promote: true } }
I20260811 12:03:22.657478 21174 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 37c923a543d6418694238c016281ad2c (127.9.221.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } peers { permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: NON_VOTER last_known_addr { host: "127.9.221.189" port: 37829 } attrs { promote: true } } }
I20260811 12:03:22.658592 21086 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:03:22.663709 21086 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c [term 1 FOLLOWER]: Refusing update from remote peer 5b972c76ffa8496faf82371b3f1523f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.221.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.188.keytab.
Entry for principal oryx/127.9.221.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.188.keytab.
I20260811 12:03:22.664224 21174 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.9.221.189:37829
I20260811 12:03:22.665158 21021 master.cc:537] Master@127.9.221.189:37829 shutting down...
I20260811 12:03:22.665889 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.188:34403
--webserver_interface=127.9.221.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.188.keytab
--principal=oryx/127.9.221.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:37859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:38537,127.9.221.189:37829,127.9.221.188:34403
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:03:22.674620 21172 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: NON_VOTER last_known_addr { host: "127.9.221.189" port: 37829 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 12:03:22.679443 20304 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260811 12:03:22.688894 20304 consensus_peers.cc:487] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 -> Peer 37c923a543d6418694238c016281ad2c (127.9.221.189:37829): Couldn't send request to peer 37c923a543d6418694238c016281ad2c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 12:03:22.689795 21175 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "5b972c76ffa8496faf82371b3f1523f7" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } peers { permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: NON_VOTER last_known_addr { host: "127.9.221.189" port: 37829 } attrs { promote: true } } }
I20260811 12:03:22.690122 21175 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 [sys.catalog]: This master's current role is: LEADER
I20260811 12:03:22.701818 21021 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:03:22.702275 21021 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:03:22.702466 21021 tablet_replica.cc:330] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c: stopping tablet replica
I20260811 12:03:22.710053 21021 master.cc:552] Master@127.9.221.189:37829 shutdown complete.
I20260811 12:03:22.710189 21021 master_runner.cc:305] Clearing existing system tablet
I20260811 12:03:22.710695 21021 fs_manager.cc:359] Metadata directory not provided
I20260811 12:03:22.710862 21021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:03:22.713819 21021 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260811 12:03:22.715510 21185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:22.715885 21021 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:03:22.716295 21021 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260811 12:03:22.716436 21021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-1/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-1/wal
uuid: "37c923a543d6418694238c016281ad2c"
format_stamp: "Formatted at 2026-08-11 12:03:22 on dist-test-slave-mjjr"
server_key: "bdac6b2233a779a1e322bec1b33c02ca"
server_key_iv: "659f7a5e6e416b28ee05313b8f4b0f83"
server_key_version: "encryptionkey@0"
I20260811 12:03:22.721462 21021 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:03:22.729024 21021 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 12:03:22.729319 21021 log.cc:1196] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260811 12:03:22.729869 21021 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c: Deleting consensus metadata
I20260811 12:03:22.730239 21021 master_runner.cc:315] Copying system tablet from 127.9.221.190:38537
I20260811 12:03:22.733588 21021 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c: tablet copy: Beginning tablet copy session from remote peer at address 127.9.221.190:38537
I20260811 12:03:22.755340 21131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:22.734731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56330 (local address 127.9.221.190:38537)
0811 12:03:22.734933 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:22.734942 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:22.740328 (+ 5386us) server_negotiation.cc:407] Connection header received
0811 12:03:22.740567 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:22.740573 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:22.740665 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:22.740786 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:22.741956 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:22.743083 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:22.745068 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:22.745742 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:22.745894 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:22.748437 (+ 2543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:22.748444 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:22.748448 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:22.748490 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:22.752249 (+ 3759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:22.753060 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:22.753066 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:22.753070 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:22.753225 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:22.753685 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:22.753690 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:22.753694 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:22.754059 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:22.754217 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:22.754710 (+ 493us) server_negotiation.cc:299] Negotiation successful
0811 12:03:22.754939 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 12:03:22.755348 21196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:22.734688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:38537 (local address 127.0.0.1:56330)
0811 12:03:22.740155 (+ 5467us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:22.740181 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:22.740382 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:22.740839 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:22.740843 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:22.741449 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:22.741774 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:22.741794 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:22.743274 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:22.743279 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:22.744902 (+ 1623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:22.744917 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:22.745938 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:22.745944 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:22.746127 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:22.747021 (+ 894us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:22.747043 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:22.748197 (+ 1154us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:22.752406 (+ 4209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:22.752411 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:22.752415 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:22.752914 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:22.753361 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:22.753370 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:22.753373 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:22.753555 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:22.754201 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:22.754206 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:22.754400 (+ 194us) client_negotiation.cc:769] Sending connection context
0811 12:03:22.754571 (+ 171us) client_negotiation.cc:240] Negotiation successful
0811 12:03:22.754887 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5293,"thread_start_us":5138,"threads_started":1}
I20260811 12:03:22.756853 20360 tablet_copy_service.cc:140] P 5b972c76ffa8496faf82371b3f1523f7: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 37c923a543d6418694238c016281ad2c ({username='oryx', principal='oryx/127.9.221.189@KRBTEST.COM'} at 127.0.0.1:56330)
I20260811 12:03:22.757292 20360 tablet_copy_service.cc:161] P 5b972c76ffa8496faf82371b3f1523f7: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 37c923a543d6418694238c016281ad2c at {username='oryx', principal='oryx/127.9.221.189@KRBTEST.COM'} at 127.0.0.1:56330: session id = 37c923a543d6418694238c016281ad2c-00000000000000000000000000000000
I20260811 12:03:22.762208 20360 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 12:03:22.765089 21021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 12:03:22.772719 21021 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c: tablet copy: Starting download of 0 data blocks...
I20260811 12:03:22.773192 21021 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c: tablet copy: Starting download of 1 WAL segments...
I20260811 12:03:22.776949 21021 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 12:03:22.781651 20360 tablet_copy_service.cc:342] P 5b972c76ffa8496faf82371b3f1523f7: Request end of tablet copy session 37c923a543d6418694238c016281ad2c-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.9.221.189@KRBTEST.COM'} at 127.0.0.1:56330
I20260811 12:03:22.781908 20360 tablet_copy_service.cc:433] P 5b972c76ffa8496faf82371b3f1523f7: ending tablet copy session 37c923a543d6418694238c016281ad2c-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 37c923a543d6418694238c016281ad2c
I20260811 12:03:22.794401 21021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:03:22.797302 21197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:22.801367 21198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:03:22.808998 21021 server_base.cc:1034] running on GCE node
W20260811 12:03:22.809619 21202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:03:22.810047 21021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:03:22.810755 21021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:03:22.811964 21021 hybrid_clock.cc:648] HybridClock initialized: now 1786449802811881 us; error 70 us; skew 500 ppm
I20260811 12:03:22.812186 21021 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:03:22 dist-test-slave-mjjr krb5kdc[20261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449802, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.221.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:03:22.814074 21021 init.cc:377] Logged in from keytab as oryx/127.9.221.189@KRBTEST.COM (short username oryx)
I20260811 12:03:22.815405 21021 webserver.cc:460] Webserver started at http://127.9.221.189:45163/ using document root <none> and password file <none>
I20260811 12:03:22.815877 21021 fs_manager.cc:359] Metadata directory not provided
I20260811 12:03:22.815970 21021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:03:22.818498 21021 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.002s
I20260811 12:03:22.820222 21217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:22.820686 21021 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260811 12:03:22.820854 21021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-1/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-1/wal
uuid: "37c923a543d6418694238c016281ad2c"
format_stamp: "Formatted at 2026-08-11 12:03:22 on dist-test-slave-mjjr"
server_key: "bdac6b2233a779a1e322bec1b33c02ca"
server_key_iv: "659f7a5e6e416b28ee05313b8f4b0f83"
server_key_version: "encryptionkey@0"
I20260811 12:03:22.821024 21021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:03:22.841964 21021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:03:22.845604 21021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:03:22.875051 21021 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.189:37829
I20260811 12:03:22.875074 21298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.189:37829 every 8 connection(s)
I20260811 12:03:22.876320 21021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-1/data/info.pb
I20260811 12:03:22.883332 21299 sys_catalog.cc:263] Verifying existing consensus state
I20260811 12:03:22.885984 21299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c: Bootstrap starting.
I20260811 12:03:22.899857 21299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 12:03:22.900388 21299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c: Bootstrap complete.
I20260811 12:03:22.901223 21299 raft_consensus.cc:348] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } peers { permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: NON_VOTER last_known_addr { host: "127.9.221.189" port: 37829 } attrs { promote: true } }
I20260811 12:03:22.901470 21299 raft_consensus.cc:729] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 37c923a543d6418694238c016281ad2c, State: Initialized, Role: LEARNER
I20260811 12:03:22.901752 21299 consensus_queue.cc:260] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } peers { permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: NON_VOTER last_known_addr { host: "127.9.221.189" port: 37829 } attrs { promote: true } }
I20260811 12:03:22.902583 21303 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } peers { permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: NON_VOTER last_known_addr { host: "127.9.221.189" port: 37829 } attrs { promote: true } } }
I20260811 12:03:22.902910 21299 sys_catalog.cc:564] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:03:22.902971 21303 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c [sys.catalog]: This master's current role is: LEARNER
I20260811 12:03:22.913882 21315 catalog_manager.cc:1259] Loaded cluster ID: 2bf09098d4e5453ea97c6c9ed333b76a
I20260811 12:03:22.914083 21315 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c: loading cluster ID for follower catalog manager: success
I20260811 12:03:22.917620 21315 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c: acquiring CA information for follower catalog manager: success
I20260811 12:03:22.920162 21315 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 12:03:22.985471 21012 heartbeater.cc:503] Master 127.9.221.190:38537 was elected leader, sending a full tablet report...
W20260811 12:03:23.120108 21177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:03:23.120584 21177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:03:23.120939 21177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:03:23.132952 21177 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:03:23.133208 21177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:03:23.133351 21177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:03:23.133450 21177 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:03:23.133558 21177 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:03:23.148532 21177 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:37859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:38537,127.9.221.189:37829,127.9.221.188:34403
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.188.keytab
--principal=oryx/127.9.221.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.188:34403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.221.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:03:23.150375 21177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:03:23.152118 21177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:03:23.171255 21324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:23.171041 21325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:23.171419 21328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:03:23.173868 21177 server_base.cc:1034] running on GCE node
I20260811 12:03:23.174736 21177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:03:23.176349 21177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:03:23.177645 21177 hybrid_clock.cc:648] HybridClock initialized: now 1786449803177571 us; error 53 us; skew 500 ppm
I20260811 12:03:23.178119 21177 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:03:23 dist-test-slave-mjjr krb5kdc[20261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449803, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.221.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:03:23.183132 21177 init.cc:377] Logged in from keytab as oryx/127.9.221.188@KRBTEST.COM (short username oryx)
I20260811 12:03:23.185578 21177 webserver.cc:460] Webserver started at http://127.9.221.188:43515/ using document root <none> and password file <none>
I20260811 12:03:23.186605 21177 fs_manager.cc:359] Metadata directory not provided
I20260811 12:03:23.186869 21177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:03:23.187314 21177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:03:23.192117 21177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-2/data/instance:
uuid: "923b96c3241c4a5ab57db67d4d33856b"
format_stamp: "Formatted at 2026-08-11 12:03:23 on dist-test-slave-mjjr"
server_key: "fb47a3cc767ce9f3b09fb77d0d589c14"
server_key_iv: "a90810ec19004ce2a26fab9ea8cdbead"
server_key_version: "encryptionkey@0"
I20260811 12:03:23.193122 21177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-2/wal/instance:
uuid: "923b96c3241c4a5ab57db67d4d33856b"
format_stamp: "Formatted at 2026-08-11 12:03:23 on dist-test-slave-mjjr"
server_key: "fb47a3cc767ce9f3b09fb77d0d589c14"
server_key_iv: "a90810ec19004ce2a26fab9ea8cdbead"
server_key_version: "encryptionkey@0"
I20260811 12:03:23.200096 21177 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 12:03:23.205708 21343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:23.207239 21177 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.001s
I20260811 12:03:23.207510 21177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-2/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-2/wal
uuid: "923b96c3241c4a5ab57db67d4d33856b"
format_stamp: "Formatted at 2026-08-11 12:03:23 on dist-test-slave-mjjr"
server_key: "fb47a3cc767ce9f3b09fb77d0d589c14"
server_key_iv: "a90810ec19004ce2a26fab9ea8cdbead"
server_key_version: "encryptionkey@0"
I20260811 12:03:23.207947 21177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:03:23.234012 21177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:03:23.237646 21350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:23.227961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36422 (local address 127.9.221.189:37829)
0811 12:03:23.229384 (+ 1423us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:23.229389 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:23.229406 (+ 17us) server_negotiation.cc:407] Connection header received
0811 12:03:23.229468 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:23.229472 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:23.229579 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:23.229714 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:03:23.230496 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:23.232157 (+ 1661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:23.234978 (+ 2821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:23.236229 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:23.236336 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:23.236512 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:23.237054 (+ 542us) server_negotiation.cc:299] Negotiation successful
0811 12:03:23.237224 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1268,"thread_start_us":1089,"threads_started":1}
I20260811 12:03:23.237703 21349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:23.227635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:37829 (local address 127.0.0.1:36422)
0811 12:03:23.228475 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:23.228498 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:23.228634 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:23.229788 (+ 1154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:23.229794 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:23.229847 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:03:23.230327 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:23.230343 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:23.232317 (+ 1974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:23.232322 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:23.234729 (+ 2407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:23.234756 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:23.236483 (+ 1727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:23.236493 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:23.236821 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:23.236886 (+ 65us) client_negotiation.cc:769] Sending connection context
0811 12:03:23.237005 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 12:03:23.237209 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":716,"thread_start_us":593,"threads_started":1}
I20260811 12:03:23.239351 21255 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260811 12:03:23.240332 21303 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b972c76ffa8496faf82371b3f1523f7. Latest consensus state: current_term: 1 leader_uuid: "5b972c76ffa8496faf82371b3f1523f7" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } peers { permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: NON_VOTER last_known_addr { host: "127.9.221.189" port: 37829 } attrs { promote: true } } }
I20260811 12:03:23.240782 21303 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c [sys.catalog]: This master's current role is: LEARNER
I20260811 12:03:23.241324 21177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:03:23.241719 21177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:03:23.290163 21177 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.188:34403
I20260811 12:03:23.290437 21418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.188:34403 every 8 connection(s)
I20260811 12:03:23.292111 21177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-2/data/info.pb
I20260811 12:03:23.299139 21419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:03:23.301939 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 21177
I20260811 12:03:23.315351 21419 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:03:23 dist-test-slave-mjjr krb5kdc[20261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.221.188@KRBTEST.COM
Aug 11 12:03:23 dist-test-slave-mjjr krb5kdc[20261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449803, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.221.188@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM
I20260811 12:03:23.339339 21422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:23.304906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41230 (local address 127.9.221.188:34403)
0811 12:03:23.306625 (+ 1719us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:23.306640 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:23.306709 (+ 69us) server_negotiation.cc:407] Connection header received
0811 12:03:23.310740 (+ 4031us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:23.310785 (+ 45us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:23.311397 (+ 612us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:23.312019 (+ 622us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:23.314357 (+ 2338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:23.317432 (+ 3075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:23.319229 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:23.321015 (+ 1786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:23.321284 (+ 269us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:23.329068 (+ 7784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:23.329078 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:23.329090 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:23.329128 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:23.334176 (+ 5048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:23.334983 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:23.334992 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:23.335006 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:23.335202 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:23.335683 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:23.335688 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:23.335696 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:23.336514 (+ 818us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:23.337004 (+ 490us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:23.337532 (+ 528us) server_negotiation.cc:299] Negotiation successful
0811 12:03:23.337993 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1378,"thread_start_us":948,"threads_started":1}
I20260811 12:03:23.344058 21423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:23.319272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56336 (local address 127.9.221.190:38537)
0811 12:03:23.320505 (+ 1233us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:23.320512 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:23.321959 (+ 1447us) server_negotiation.cc:407] Connection header received
0811 12:03:23.322248 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:23.322255 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:23.322355 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:23.322500 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:23.325502 (+ 3002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:23.326651 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:23.329086 (+ 2435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:23.329697 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:23.329841 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:23.334936 (+ 5095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:23.334941 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:23.334944 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:23.334976 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:23.339060 (+ 4084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:23.340239 (+ 1179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:23.340245 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:23.340248 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:23.340458 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:23.342367 (+ 1909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:23.342371 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:23.342373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:23.342710 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:23.342903 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:23.343360 (+ 457us) server_negotiation.cc:299] Negotiation successful
0811 12:03:23.343540 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1023,"thread_start_us":893,"threads_started":1}
I20260811 12:03:23.344098 21424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:23.319401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:38537 (local address 127.0.0.1:56336)
0811 12:03:23.321793 (+ 2392us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:23.321841 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:23.322094 (+ 253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:23.322638 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:23.322654 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:23.324426 (+ 1772us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:23.325311 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:23.325328 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:23.326872 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:23.326887 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:23.328910 (+ 2023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:23.328931 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:23.330070 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:23.330089 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:23.330374 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:23.331204 (+ 830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:23.331224 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:23.334694 (+ 3470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:23.339201 (+ 4507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:23.339212 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:23.339219 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:23.340106 (+ 887us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:23.341997 (+ 1891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:23.342000 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:23.342003 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:23.342195 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:23.342824 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:23.342829 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:23.343045 (+ 216us) client_negotiation.cc:769] Sending connection context
0811 12:03:23.343245 (+ 200us) client_negotiation.cc:240] Negotiation successful
0811 12:03:23.343479 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1782,"thread_start_us":1354,"threads_started":1}
I20260811 12:03:23.345551 21419 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.221.189" port: 37829 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:03:23 dist-test-slave-mjjr krb5kdc[20261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449803, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.221.188@KRBTEST.COM for oryx/127.9.221.189@KRBTEST.COM
I20260811 12:03:23.362313 21424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:23.346340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:37829 (local address 127.0.0.1:36438)
0811 12:03:23.346659 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:23.346675 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:23.346750 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:23.347009 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:23.347012 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:23.347487 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:23.347784 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:23.347795 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:23.349087 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:23.349090 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:23.350408 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:23.350419 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:23.351330 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:23.351335 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:23.351584 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:23.352362 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:23.352379 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:23.355085 (+ 2706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:23.359233 (+ 4148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:23.359237 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:23.359240 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:23.360122 (+ 882us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:23.360641 (+ 519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:23.360644 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:23.360647 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:23.360909 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:23.361512 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:23.361515 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:23.361640 (+ 125us) client_negotiation.cc:769] Sending connection context
0811 12:03:23.361736 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 12:03:23.361995 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90}
I20260811 12:03:23.362490 21350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:23.346516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36438 (local address 127.9.221.189:37829)
0811 12:03:23.346731 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:23.346736 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:23.346753 (+ 17us) server_negotiation.cc:407] Connection header received
0811 12:03:23.346828 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:23.346832 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:23.346891 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:23.346980 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:23.347911 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:23.348906 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:23.350534 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:23.351168 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:23.351271 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:23.355299 (+ 4028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:23.355304 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:23.355308 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:23.355340 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:23.359052 (+ 3712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:23.360245 (+ 1193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:23.360250 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:23.360253 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:23.360496 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:23.361039 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:23.361042 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:23.361045 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:23.361384 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:23.361722 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:23.361858 (+ 136us) server_negotiation.cc:299] Negotiation successful
0811 12:03:23.362094 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 12:03:23.363520 21419 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.221.188" port: 34403 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:03:23 dist-test-slave-mjjr krb5kdc[20261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449803, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.221.188@KRBTEST.COM for oryx/127.9.221.188@KRBTEST.COM
I20260811 12:03:23.379807 21422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:23.364679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41232 (local address 127.9.221.188:34403)
0811 12:03:23.364832 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:23.364839 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:23.364858 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:03:23.364898 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:23.364903 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:23.364963 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:23.365058 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:23.366013 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:23.366970 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:23.368378 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:23.369019 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:23.369179 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:23.373036 (+ 3857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:23.373040 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:23.373043 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:23.373071 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:23.376726 (+ 3655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:23.377531 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:23.377535 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:23.377537 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:23.377707 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:23.378202 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:23.378205 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:23.378207 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:23.378551 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:23.378660 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:23.379137 (+ 477us) server_negotiation.cc:299] Negotiation successful
0811 12:03:23.379334 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":54}
I20260811 12:03:23.379837 21424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:23.364282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:34403 (local address 127.0.0.1:41232)
0811 12:03:23.364653 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:23.364670 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:23.364789 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:23.365073 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:23.365077 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:23.365601 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:23.365873 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:23.365882 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:23.367096 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:23.367104 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:23.368246 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:23.368258 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:23.369239 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:23.369246 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:23.369660 (+ 414us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:23.370257 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:23.370273 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:23.372870 (+ 2597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:23.376866 (+ 3996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:23.376870 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:23.376872 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:23.377433 (+ 561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:23.377806 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:23.377809 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:23.377811 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:23.378105 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:23.378682 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:23.378686 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:23.378873 (+ 187us) client_negotiation.cc:769] Sending connection context
0811 12:03:23.379023 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 12:03:23.379206 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98}
I20260811 12:03:23.386404 21419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b: Bootstrap starting.
I20260811 12:03:23.391848 21419 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b: Neither blocks nor log segments found. Creating new log.
I20260811 12:03:23.394055 21419 log.cc:824] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b: Log is configured to *not* fsync() on all Append() calls
I20260811 12:03:23.398281 21419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b: No bootstrap required, opened a new log
I20260811 12:03:23.409752 21419 raft_consensus.cc:348] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } peers { permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 37829 } } peers { permanent_uuid: "923b96c3241c4a5ab57db67d4d33856b" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 34403 } }
I20260811 12:03:23.410375 21419 raft_consensus.cc:374] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:03:23.410656 21419 raft_consensus.cc:729] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 923b96c3241c4a5ab57db67d4d33856b, State: Initialized, Role: FOLLOWER
I20260811 12:03:23.411327 21419 consensus_queue.cc:260] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } peers { permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 37829 } } peers { permanent_uuid: "923b96c3241c4a5ab57db67d4d33856b" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 34403 } }
I20260811 12:03:23.413105 21426 sys_catalog.cc:455] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } peers { permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 37829 } } peers { permanent_uuid: "923b96c3241c4a5ab57db67d4d33856b" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 34403 } } }
I20260811 12:03:23.413851 21426 sys_catalog.cc:458] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:03:23.414264 21419 sys_catalog.cc:564] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [sys.catalog]: configured and running, proceeding with master startup.
W20260811 12:03:23.432204 21449 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 12:03:23.432425 21449 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 12:03:23.436913 21177 master_runner.cc:428] Detected that this master 923b96c3241c4a5ab57db67d4d33856b is joining an existing cluster
I20260811 12:03:23.437131 21177 master_runner.cc:432] Initiating AddMaster RPC to add 127.9.221.188:34403
I20260811 12:03:23.447028 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 20431
Aug 11 12:03:23 dist-test-slave-mjjr krb5kdc[20261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.221.129@KRBTEST.COM
I20260811 12:03:23.484072 21350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:23.444776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36446 (local address 127.9.221.189:37829)
0811 12:03:23.445439 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:23.445445 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:23.458216 (+ 12771us) server_negotiation.cc:407] Connection header received
0811 12:03:23.458402 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:23.458408 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:23.458485 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:23.458567 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:23.464433 (+ 5866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:23.465985 (+ 1552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:23.469863 (+ 3878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:23.470867 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:23.470997 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:23.476300 (+ 5303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:23.476310 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:23.476317 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:23.476379 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:23.480355 (+ 3976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:23.481329 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:23.481337 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:23.481340 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:23.481512 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:23.482056 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:23.482062 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:23.482066 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:23.482445 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:23.482589 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:23.483310 (+ 721us) server_negotiation.cc:299] Negotiation successful
0811 12:03:23.483566 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539}
I20260811 12:03:23.485599 21459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:23.445400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:37829 (local address 127.0.0.1:36446)
0811 12:03:23.458072 (+ 12672us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:23.458096 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:23.458245 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:23.458822 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:23.458829 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:23.461184 (+ 2355us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:23.464170 (+ 2986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:23.464207 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:23.466410 (+ 2203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:23.466415 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:23.469652 (+ 3237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:23.469669 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:23.471414 (+ 1745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:23.471423 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:23.471751 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:23.474167 (+ 2416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:23.474199 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:23.475973 (+ 1774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:23.480487 (+ 4514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:23.480495 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:23.480502 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:23.481168 (+ 666us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:23.481666 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:23.481672 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:23.481676 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:23.481899 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:23.482689 (+ 790us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:23.482694 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:23.482962 (+ 268us) client_negotiation.cc:769] Sending connection context
0811 12:03:23.484600 (+ 1638us) client_negotiation.cc:240] Negotiation successful
0811 12:03:23.484968 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12420,"thread_start_us":11264,"threads_started":1}
I20260811 12:03:23.487439 21423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:23.444317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56348 (local address 127.9.221.190:38537)
0811 12:03:23.445018 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:23.445026 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:23.452827 (+ 7801us) server_negotiation.cc:407] Connection header received
0811 12:03:23.455003 (+ 2176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:23.455012 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:23.455185 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:23.455278 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:23.465077 (+ 9799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:23.466882 (+ 1805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:23.472281 (+ 5399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:23.473138 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:23.473297 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:23.477875 (+ 4578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:23.477882 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:23.477888 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:23.477927 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:23.482422 (+ 4495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:23.483397 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:23.483404 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:23.483409 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:23.483576 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:23.484647 (+ 1071us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:23.484655 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:23.484661 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:23.485090 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:23.485302 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:23.486638 (+ 1336us) server_negotiation.cc:299] Negotiation successful
0811 12:03:23.486938 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530}
I20260811 12:03:23.488794 21458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:23.444601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:38537 (local address 127.0.0.1:56348)
0811 12:03:23.452670 (+ 8069us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:23.452697 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:23.454658 (+ 1961us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:23.455628 (+ 970us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:23.455636 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:23.463422 (+ 7786us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:23.464908 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:23.464926 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:23.467138 (+ 2212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:23.467143 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:23.472095 (+ 4952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:23.472114 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:23.473375 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:23.473383 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:23.473575 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:23.476243 (+ 2668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:23.476271 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:23.477627 (+ 1356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:23.482593 (+ 4966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:23.482598 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:23.482602 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:23.483250 (+ 648us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:23.484142 (+ 892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:23.484146 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:23.484150 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:23.484462 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:23.485687 (+ 1225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:23.485700 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:23.486138 (+ 438us) client_negotiation.cc:769] Sending connection context
0811 12:03:23.487673 (+ 1535us) client_negotiation.cc:240] Negotiation successful
0811 12:03:23.488044 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7808,"thread_start_us":8251,"threads_started":1}
I20260811 12:03:23.490651 21461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:23.448681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36604 (local address 127.9.221.129:34021)
0811 12:03:23.457931 (+ 9250us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:23.457937 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:23.457961 (+ 24us) server_negotiation.cc:407] Connection header received
0811 12:03:23.458040 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:23.458046 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:23.458291 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:23.461452 (+ 3161us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:23.461819 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:23.464118 (+ 2299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:23.466905 (+ 2787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:23.468354 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:23.474326 (+ 5972us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:23.477416 (+ 3090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:23.477432 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:23.477451 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:23.477506 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:23.485854 (+ 8348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:23.486899 (+ 1045us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:23.486913 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:23.486920 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:23.487106 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:23.487954 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:23.487957 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:23.487960 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:23.488359 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:23.488699 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:23.489907 (+ 1208us) server_negotiation.cc:299] Negotiation successful
0811 12:03:23.490193 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9122,"thread_start_us":8580,"threads_started":1}
I20260811 12:03:23.492197 20602 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:03:23.494465 21422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:23.454109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41248 (local address 127.9.221.188:34403)
0811 12:03:23.454311 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:23.454316 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:23.467005 (+ 12689us) server_negotiation.cc:407] Connection header received
0811 12:03:23.467736 (+ 731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:23.467742 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:23.467827 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:23.467901 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:23.472667 (+ 4766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:23.473690 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:23.478725 (+ 5035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:23.479457 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:23.479566 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:23.482933 (+ 3367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:23.482940 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:23.482945 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:23.482983 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:23.488452 (+ 5469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:23.489637 (+ 1185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:23.489643 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:23.489647 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:23.489797 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:23.490272 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:23.490277 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:23.490280 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:23.490687 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:23.491128 (+ 441us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:23.492211 (+ 1083us) server_negotiation.cc:299] Negotiation successful
0811 12:03:23.493984 (+ 1773us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260811 12:03:23.494993 21354 tablet.cc:2369] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:03:23.496043 21462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:23.454278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:34403 (local address 127.0.0.1:41248)
0811 12:03:23.466904 (+ 12626us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:23.466936 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:23.467550 (+ 614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:23.468147 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:23.468151 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:23.469137 (+ 986us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:23.472479 (+ 3342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:23.472493 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:23.474178 (+ 1685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:23.474183 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:23.478484 (+ 4301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:23.478501 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:23.479898 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:23.479907 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:23.480144 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:23.481449 (+ 1305us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:23.481473 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:23.482695 (+ 1222us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:23.488631 (+ 5936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:23.488638 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:23.488644 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:23.489507 (+ 863us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:23.489944 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:23.489948 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:23.489951 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:23.490114 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:23.491197 (+ 1083us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:23.491201 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:23.491388 (+ 187us) client_negotiation.cc:769] Sending connection context
0811 12:03:23.494936 (+ 3548us) client_negotiation.cc:240] Negotiation successful
0811 12:03:23.495438 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12234,"thread_start_us":12758,"threads_started":1}
Aug 11 12:03:23 dist-test-slave-mjjr krb5kdc[20261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449800, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.221.190@KRBTEST.COM for oryx/127.9.221.188@KRBTEST.COM
I20260811 12:03:23.515460 21349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:23.496283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:34403 (local address 127.0.0.1:41252)
0811 12:03:23.496672 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:23.496692 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:23.496799 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:23.497577 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:23.497583 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:23.498279 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:23.498510 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:23.498523 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:23.500031 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:23.500040 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:23.501116 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:23.501130 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:23.502898 (+ 1768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:23.502905 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:23.503123 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:23.504167 (+ 1044us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:23.504201 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:23.507599 (+ 3398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:23.512075 (+ 4476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:23.512081 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:23.512085 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:23.512633 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:23.513212 (+ 579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:23.513217 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:23.513221 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:23.513426 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:23.514371 (+ 945us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:23.514377 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:23.514563 (+ 186us) client_negotiation.cc:769] Sending connection context
0811 12:03:23.514743 (+ 180us) client_negotiation.cc:240] Negotiation successful
0811 12:03:23.515033 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253}
I20260811 12:03:23.516512 21422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:23.496958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41252 (local address 127.9.221.188:34403)
0811 12:03:23.497144 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:23.497152 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:23.497171 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:03:23.497287 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:23.497293 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:23.497387 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:23.498680 (+ 1293us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:23.498872 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:23.499888 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:23.501321 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:23.502720 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:23.502886 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:23.507892 (+ 5006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:23.507899 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:23.507905 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:23.507945 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:23.511894 (+ 3949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:23.512807 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:23.512813 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:23.512817 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:23.513054 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:23.513625 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:23.513631 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:23.513634 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:23.514144 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:23.514297 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:23.515826 (+ 1529us) server_negotiation.cc:299] Negotiation successful
0811 12:03:23.516063 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 12:03:23.516778 21426 raft_consensus.cc:482] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:03:23.517174 21426 raft_consensus.cc:504] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } peers { permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 37829 } } peers { permanent_uuid: "923b96c3241c4a5ab57db67d4d33856b" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 34403 } }
I20260811 12:03:23.519037 21426 leader_election.cc:288] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b972c76ffa8496faf82371b3f1523f7 (127.9.221.190:38537), 37c923a543d6418694238c016281ad2c (127.9.221.189:37829)
I20260811 12:03:23.519277 20326 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.9.221.188:34403: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "923b96c3241c4a5ab57db67d4d33856b"
member_type: NON_VOTER
last_known_addr {
host: "127.9.221.188"
port: 34403
}
attrs {
promote: true
}
}
dest_uuid: "5b972c76ffa8496faf82371b3f1523f7"
cas_config_opid_index: 5
I20260811 12:03:23.520418 20326 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } peers { permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: NON_VOTER last_known_addr { host: "127.9.221.189" port: 37829 } attrs { promote: true } } peers { permanent_uuid: "923b96c3241c4a5ab57db67d4d33856b" member_type: NON_VOTER last_known_addr { host: "127.9.221.188" port: 34403 } attrs { promote: true } }
I20260811 12:03:23.522233 21255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "923b96c3241c4a5ab57db67d4d33856b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37c923a543d6418694238c016281ad2c" is_pre_election: true
I20260811 12:03:23.523804 20341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "923b96c3241c4a5ab57db67d4d33856b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b972c76ffa8496faf82371b3f1523f7" is_pre_election: true
I20260811 12:03:23.523890 21348 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 923b96c3241c4a5ab57db67d4d33856b (127.9.221.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } peers { permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: NON_VOTER last_known_addr { host: "127.9.221.189" port: 37829 } attrs { promote: true } } peers { permanent_uuid: "923b96c3241c4a5ab57db67d4d33856b" member_type: NON_VOTER last_known_addr { host: "127.9.221.188" port: 34403 } attrs { promote: true } } }
I20260811 12:03:23.528867 21347 leader_election.cc:302] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 923b96c3241c4a5ab57db67d4d33856b; no voters: 37c923a543d6418694238c016281ad2c, 5b972c76ffa8496faf82371b3f1523f7
I20260811 12:03:23.529577 21426 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 12:03:23.555022 21385 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:03:23.562136 21385 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [term 1 FOLLOWER]: Refusing update from remote peer 5b972c76ffa8496faf82371b3f1523f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 12:03:23.566012 21255 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c [term 1 LEARNER]: Refusing update from remote peer 5b972c76ffa8496faf82371b3f1523f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 12:03:23.572623 21348 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.9.221.188:34403
I20260811 12:03:23.572402 21470 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "923b96c3241c4a5ab57db67d4d33856b" member_type: NON_VOTER last_known_addr { host: "127.9.221.188" port: 34403 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.001s
I20260811 12:03:23.573647 21177 master.cc:537] Master@127.9.221.188:34403 shutting down...
I20260811 12:03:23.574575 21469 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: NON_VOTER last_known_addr { host: "127.9.221.189" port: 37829 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260811 12:03:23.576290 21465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "5b972c76ffa8496faf82371b3f1523f7" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } peers { permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: NON_VOTER last_known_addr { host: "127.9.221.189" port: 37829 } attrs { promote: true } } peers { permanent_uuid: "923b96c3241c4a5ab57db67d4d33856b" member_type: NON_VOTER last_known_addr { host: "127.9.221.188" port: 34403 } attrs { promote: true } } }
I20260811 12:03:23.577066 21465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 [sys.catalog]: This master's current role is: LEADER
W20260811 12:03:23.596585 20309 consensus_peers.cc:487] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 -> Peer 923b96c3241c4a5ab57db67d4d33856b (127.9.221.188:34403): Couldn't send request to peer 923b96c3241c4a5ab57db67d4d33856b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 12:03:23.596060 21255 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 923b96c3241c4a5ab57db67d4d33856b (127.9.221.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } peers { permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: NON_VOTER last_known_addr { host: "127.9.221.189" port: 37829 } attrs { promote: true } } peers { permanent_uuid: "923b96c3241c4a5ab57db67d4d33856b" member_type: NON_VOTER last_known_addr { host: "127.9.221.188" port: 34403 } attrs { promote: true } } }
I20260811 12:03:23.602309 21303 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5b972c76ffa8496faf82371b3f1523f7" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } peers { permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: NON_VOTER last_known_addr { host: "127.9.221.189" port: 37829 } attrs { promote: true } } peers { permanent_uuid: "923b96c3241c4a5ab57db67d4d33856b" member_type: NON_VOTER last_known_addr { host: "127.9.221.188" port: 34403 } attrs { promote: true } } }
I20260811 12:03:23.602698 21303 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c [sys.catalog]: This master's current role is: LEARNER
I20260811 12:03:23.604075 21469 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7: attempting to promote NON_VOTER 37c923a543d6418694238c016281ad2c to VOTER
I20260811 12:03:23.605419 21469 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } peers { permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 37829 } attrs { promote: false } } peers { permanent_uuid: "923b96c3241c4a5ab57db67d4d33856b" member_type: NON_VOTER last_known_addr { host: "127.9.221.188" port: 34403 } attrs { promote: true } }
I20260811 12:03:23.606714 21177 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:03:23.607125 21177 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:03:23.607314 21177 tablet_replica.cc:330] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b: stopping tablet replica
W20260811 12:03:23.609442 20309 consensus_peers.cc:487] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 -> Peer 923b96c3241c4a5ab57db67d4d33856b (127.9.221.188:34403): Couldn't send request to peer 923b96c3241c4a5ab57db67d4d33856b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 12:03:23.610877 21255 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c [term 1 LEARNER]: Refusing update from remote peer 5b972c76ffa8496faf82371b3f1523f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 12:03:23.611539 21348 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 37829 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260811 12:03:23.615338 21466 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 37c923a543d6418694238c016281ad2c (127.9.221.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } peers { permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 37829 } attrs { promote: false } } peers { permanent_uuid: "923b96c3241c4a5ab57db67d4d33856b" member_type: NON_VOTER last_known_addr { host: "127.9.221.188" port: 34403 } attrs { promote: true } } }
I20260811 12:03:23.619999 21255 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 37c923a543d6418694238c016281ad2c (127.9.221.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } peers { permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 37829 } attrs { promote: false } } peers { permanent_uuid: "923b96c3241c4a5ab57db67d4d33856b" member_type: NON_VOTER last_known_addr { host: "127.9.221.188" port: 34403 } attrs { promote: true } } }
I20260811 12:03:23.621074 21466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "5b972c76ffa8496faf82371b3f1523f7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } peers { permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 37829 } attrs { promote: false } } peers { permanent_uuid: "923b96c3241c4a5ab57db67d4d33856b" member_type: NON_VOTER last_known_addr { host: "127.9.221.188" port: 34403 } attrs { promote: true } } }
I20260811 12:03:23.621447 21466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 [sys.catalog]: This master's current role is: LEADER
I20260811 12:03:23.621750 21348 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5b972c76ffa8496faf82371b3f1523f7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } peers { permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 37829 } attrs { promote: false } } peers { permanent_uuid: "923b96c3241c4a5ab57db67d4d33856b" member_type: NON_VOTER last_known_addr { host: "127.9.221.188" port: 34403 } attrs { promote: true } } }
I20260811 12:03:23.622126 21348 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 [sys.catalog]: This master's current role is: LEADER
I20260811 12:03:23.623854 21177 master.cc:552] Master@127.9.221.188:34403 shutdown complete.
I20260811 12:03:23.623957 21177 master_runner.cc:305] Clearing existing system tablet
I20260811 12:03:23.624460 21177 fs_manager.cc:359] Metadata directory not provided
I20260811 12:03:23.624675 21177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:03:23.625697 21303 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5b972c76ffa8496faf82371b3f1523f7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } peers { permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 37829 } attrs { promote: false } } peers { permanent_uuid: "923b96c3241c4a5ab57db67d4d33856b" member_type: NON_VOTER last_known_addr { host: "127.9.221.188" port: 34403 } attrs { promote: true } } }
I20260811 12:03:23.625998 21303 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:03:23.631665 21177 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.005s sys 0.001s
I20260811 12:03:23.634298 21477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:23.634792 21177 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:03:23.635290 21177 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 12:03:23.635453 21177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-2/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-2/wal
uuid: "923b96c3241c4a5ab57db67d4d33856b"
format_stamp: "Formatted at 2026-08-11 12:03:23 on dist-test-slave-mjjr"
server_key: "fb47a3cc767ce9f3b09fb77d0d589c14"
server_key_iv: "a90810ec19004ce2a26fab9ea8cdbead"
server_key_version: "encryptionkey@0"
I20260811 12:03:23.641022 21177 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:03:23.646723 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 20431
I20260811 12:03:23.651934 21177 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 12:03:23.652196 21177 log.cc:1196] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 12:03:23.653051 21177 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b: Deleting consensus metadata
I20260811 12:03:23.653364 21177 master_runner.cc:315] Copying system tablet from 127.9.221.190:38537
I20260811 12:03:23.661784 21177 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b: tablet copy: Beginning tablet copy session from remote peer at address 127.9.221.190:38537
I20260811 12:03:23.672467 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.129:34021
--local_ip_for_outbound_sockets=127.9.221.129
--tserver_master_addrs=127.9.221.190:38537
--webserver_port=44775
--webserver_interface=127.9.221.129
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.129.keytab
--principal=oryx/127.9.221.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:37859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.221.190:38537,127.9.221.189:37829
--tserver_master_addrs=127.9.221.190:38537,127.9.221.189:37829,127.9.221.188:34403 with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:03:23.686134 21489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:23.662748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:38537 (local address 127.0.0.1:56352)
0811 12:03:23.668166 (+ 5418us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:23.668190 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:23.668412 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:23.668887 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:23.668892 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:23.669598 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:23.669976 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:23.669991 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:23.671670 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:23.671677 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:23.673195 (+ 1518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:23.673220 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:23.674517 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:23.674524 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:23.674740 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:23.675724 (+ 984us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:23.675751 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:23.677270 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:23.682612 (+ 5342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:23.682618 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:23.682622 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:23.683290 (+ 668us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:23.683779 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:23.683787 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:23.683793 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:23.683983 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:23.684800 (+ 817us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:23.684807 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:23.685008 (+ 201us) client_negotiation.cc:769] Sending connection context
0811 12:03:23.685184 (+ 176us) client_negotiation.cc:240] Negotiation successful
0811 12:03:23.685553 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5240,"thread_start_us":5097,"threads_started":1}
I20260811 12:03:23.686295 21423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:23.662791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56352 (local address 127.9.221.190:38537)
0811 12:03:23.663048 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:23.663056 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:23.668332 (+ 5276us) server_negotiation.cc:407] Connection header received
0811 12:03:23.668576 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:23.668586 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:23.668700 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:23.668828 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:23.670167 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:23.671448 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:23.673377 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:23.674257 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:23.674405 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:23.677557 (+ 3152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:23.677566 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:23.677573 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:23.677618 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:23.682453 (+ 4835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:23.683435 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:23.683443 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:23.683449 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:23.683609 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:23.684133 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:23.684139 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:23.684143 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:23.684650 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:23.684831 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:23.685439 (+ 608us) server_negotiation.cc:299] Negotiation successful
0811 12:03:23.685706 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260811 12:03:23.686993 20360 tablet_copy_service.cc:140] P 5b972c76ffa8496faf82371b3f1523f7: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 923b96c3241c4a5ab57db67d4d33856b ({username='oryx', principal='oryx/127.9.221.188@KRBTEST.COM'} at 127.0.0.1:56352)
I20260811 12:03:23.687230 20360 tablet_copy_service.cc:161] P 5b972c76ffa8496faf82371b3f1523f7: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 923b96c3241c4a5ab57db67d4d33856b at {username='oryx', principal='oryx/127.9.221.188@KRBTEST.COM'} at 127.0.0.1:56352: session id = 923b96c3241c4a5ab57db67d4d33856b-00000000000000000000000000000000
I20260811 12:03:23.692201 20360 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 12:03:23.694562 21177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 12:03:23.704469 21177 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b: tablet copy: Starting download of 0 data blocks...
I20260811 12:03:23.704864 21177 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b: tablet copy: Starting download of 1 WAL segments...
I20260811 12:03:23.707706 21177 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 12:03:23.714274 20360 tablet_copy_service.cc:342] P 5b972c76ffa8496faf82371b3f1523f7: Request end of tablet copy session 923b96c3241c4a5ab57db67d4d33856b-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.9.221.188@KRBTEST.COM'} at 127.0.0.1:56352
I20260811 12:03:23.714586 20360 tablet_copy_service.cc:433] P 5b972c76ffa8496faf82371b3f1523f7: ending tablet copy session 923b96c3241c4a5ab57db67d4d33856b-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 923b96c3241c4a5ab57db67d4d33856b
I20260811 12:03:23.729032 21177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:03:23.731931 21492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:23.737452 21493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:23.749922 21495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:03:23.750350 21177 server_base.cc:1034] running on GCE node
I20260811 12:03:23.750810 21177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:03:23.751775 21177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:03:23.753032 21177 hybrid_clock.cc:648] HybridClock initialized: now 1786449803752961 us; error 43 us; skew 500 ppm
I20260811 12:03:23.753403 21177 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:03:23 dist-test-slave-mjjr krb5kdc[20261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449803, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.221.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:03:23.756352 21177 init.cc:377] Logged in from keytab as oryx/127.9.221.188@KRBTEST.COM (short username oryx)
I20260811 12:03:23.758714 21177 webserver.cc:460] Webserver started at http://127.9.221.188:36497/ using document root <none> and password file <none>
I20260811 12:03:23.759331 21177 fs_manager.cc:359] Metadata directory not provided
I20260811 12:03:23.759479 21177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:03:23.766254 21177 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.000s sys 0.005s
I20260811 12:03:23.770290 21507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:23.771068 21177 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 12:03:23.771325 21177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-2/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-2/wal
uuid: "923b96c3241c4a5ab57db67d4d33856b"
format_stamp: "Formatted at 2026-08-11 12:03:23 on dist-test-slave-mjjr"
server_key: "fb47a3cc767ce9f3b09fb77d0d589c14"
server_key_iv: "a90810ec19004ce2a26fab9ea8cdbead"
server_key_version: "encryptionkey@0"
I20260811 12:03:23.771778 21177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:03:23.788532 21177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:03:23.794459 21177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:03:23.913082 21177 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.188:34403
I20260811 12:03:23.913138 21588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.188:34403 every 8 connection(s)
I20260811 12:03:23.914180 21177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/master-2/data/info.pb
I20260811 12:03:23.921962 21589 sys_catalog.cc:263] Verifying existing consensus state
I20260811 12:03:23.927881 21589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b: Bootstrap starting.
I20260811 12:03:23.945586 21589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 12:03:23.946131 21589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b: Bootstrap complete.
I20260811 12:03:23.947057 21589 raft_consensus.cc:348] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } peers { permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 37829 } attrs { promote: false } } peers { permanent_uuid: "923b96c3241c4a5ab57db67d4d33856b" member_type: NON_VOTER last_known_addr { host: "127.9.221.188" port: 34403 } attrs { promote: true } }
I20260811 12:03:23.947326 21589 raft_consensus.cc:729] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 923b96c3241c4a5ab57db67d4d33856b, State: Initialized, Role: LEARNER
I20260811 12:03:23.947568 21589 consensus_queue.cc:260] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } peers { permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 37829 } attrs { promote: false } } peers { permanent_uuid: "923b96c3241c4a5ab57db67d4d33856b" member_type: NON_VOTER last_known_addr { host: "127.9.221.188" port: 34403 } attrs { promote: true } }
I20260811 12:03:23.949133 21589 sys_catalog.cc:564] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:03:23.948766 21592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } peers { permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 37829 } attrs { promote: false } } peers { permanent_uuid: "923b96c3241c4a5ab57db67d4d33856b" member_type: NON_VOTER last_known_addr { host: "127.9.221.188" port: 34403 } attrs { promote: true } } }
I20260811 12:03:23.949396 21592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [sys.catalog]: This master's current role is: LEARNER
I20260811 12:03:23.958801 21609 catalog_manager.cc:1259] Loaded cluster ID: 2bf09098d4e5453ea97c6c9ed333b76a
I20260811 12:03:23.958966 21609 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b: loading cluster ID for follower catalog manager: success
I20260811 12:03:23.963398 21609 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b: acquiring CA information for follower catalog manager: success
I20260811 12:03:23.965909 21609 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 12:03:24.107568 21611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:24.101616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41260 (local address 127.9.221.188:34403)
0811 12:03:24.102285 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:24.102290 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:24.102305 (+ 15us) server_negotiation.cc:407] Connection header received
0811 12:03:24.102363 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:24.102367 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:24.102440 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:24.102583 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:03:24.103026 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:24.104159 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:24.105729 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:24.106534 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:24.106680 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:24.106930 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:24.107082 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 12:03:24.107214 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":371,"threads_started":1}
I20260811 12:03:24.107545 21610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:24.101391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:34403 (local address 127.0.0.1:41260)
0811 12:03:24.101897 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:24.101923 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:24.102096 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:24.102609 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:24.102613 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:24.102640 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:03:24.102871 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:24.102879 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:24.104306 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:24.104312 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:24.105573 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:24.105586 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:24.106706 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:24.106712 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:24.106877 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:24.106922 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 12:03:24.107059 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 12:03:24.107206 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":269,"threads_started":1}
I20260811 12:03:24.108726 21550 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260811 12:03:24.109658 21592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b972c76ffa8496faf82371b3f1523f7. Latest consensus state: current_term: 1 leader_uuid: "5b972c76ffa8496faf82371b3f1523f7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } peers { permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 37829 } attrs { promote: false } } peers { permanent_uuid: "923b96c3241c4a5ab57db67d4d33856b" member_type: NON_VOTER last_known_addr { host: "127.9.221.188" port: 34403 } attrs { promote: true } } }
I20260811 12:03:24.109948 21592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [sys.catalog]: This master's current role is: LEARNER
W20260811 12:03:24.147759 21491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:03:24.148078 21491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:03:24.148250 21491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:03:24.155870 21491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:03:24.156010 21491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:03:24.156183 21491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.129
I20260811 12:03:24.166074 21491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:37859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.129.keytab
--principal=oryx/127.9.221.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.129:34021
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.221.129
--webserver_port=44775
--tserver_master_addrs=127.9.221.190:38537,127.9.221.189:37829,127.9.221.188:34403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.129
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:03:24.167421 21491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:03:24.168807 21491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:03:24.182384 21618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:24.182478 21622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:24.182482 21617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:03:24.183427 21491 server_base.cc:1034] running on GCE node
I20260811 12:03:24.184091 21491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:03:24.185014 21491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:03:24.186242 21491 hybrid_clock.cc:648] HybridClock initialized: now 1786449804186198 us; error 54 us; skew 500 ppm
I20260811 12:03:24.186573 21491 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:03:24 dist-test-slave-mjjr krb5kdc[20261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449804, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.221.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:03:24.189683 21491 init.cc:377] Logged in from keytab as oryx/127.9.221.129@KRBTEST.COM (short username oryx)
I20260811 12:03:24.191107 21491 webserver.cc:460] Webserver started at http://127.9.221.129:44775/ using document root <none> and password file <none>
I20260811 12:03:24.191855 21491 fs_manager.cc:359] Metadata directory not provided
I20260811 12:03:24.192022 21491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:03:24.197127 21491 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:03:24.199889 21632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:24.201206 21491 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:03:24.201404 21491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-0/wal
uuid: "5d3b7729ac6448e1a27b453a8f278fbc"
format_stamp: "Formatted at 2026-08-11 12:03:20 on dist-test-slave-mjjr"
server_key: "fe457c062d5f8448203fe144d6dc7e82"
server_key_iv: "60ec8dcdd1a777a36a82f9b2a18ecfd4"
server_key_version: "encryptionkey@0"
I20260811 12:03:24.202397 21491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:03:24.237692 21491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:03:24.241621 21491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:03:24.241900 21491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:03:24.242930 21491 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:03:24.244392 21491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:03:24.244490 21491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:24.244572 21491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:03:24.244626 21491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:24.316414 21491 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.129:34021
I20260811 12:03:24.316581 21806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.129:34021 every 8 connection(s)
I20260811 12:03:24.318222 21491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
I20260811 12:03:24.327368 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 21491
I20260811 12:03:24.327893 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 20617
Aug 11 12:03:24 dist-test-slave-mjjr krb5kdc[20261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449804, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.221.129@KRBTEST.COM for oryx/127.9.221.188@KRBTEST.COM
Aug 11 12:03:24 dist-test-slave-mjjr krb5kdc[20261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449804, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.221.129@KRBTEST.COM for oryx/127.9.221.189@KRBTEST.COM
Aug 11 12:03:24 dist-test-slave-mjjr krb5kdc[20261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449804, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.221.129@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM
Aug 11 12:03:24 dist-test-slave-mjjr krb5kdc[20261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.221.130@KRBTEST.COM
I20260811 12:03:24.358839 21817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:24.325676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:52221 (local address 127.9.221.190:38537)
0811 12:03:24.326432 (+ 756us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:24.326437 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:24.328718 (+ 2281us) server_negotiation.cc:407] Connection header received
0811 12:03:24.330766 (+ 2048us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:24.330776 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:24.330962 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:24.331273 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:24.334817 (+ 3544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:24.336223 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:24.340032 (+ 3809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:24.340860 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:24.340966 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:24.349127 (+ 8161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:24.349135 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:24.349140 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:24.349183 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:24.354993 (+ 5810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:24.356076 (+ 1083us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:24.356081 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:24.356086 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:24.356270 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:24.356814 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:24.356822 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:24.356825 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:24.357213 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:24.357389 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:24.358149 (+ 760us) server_negotiation.cc:299] Negotiation successful
0811 12:03:24.358413 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":661,"thread_start_us":513,"threads_started":1}
I20260811 12:03:24.359107 21813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:24.322104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:41763 (local address 127.9.221.189:37829)
0811 12:03:24.322958 (+ 854us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:24.322965 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:24.325279 (+ 2314us) server_negotiation.cc:407] Connection header received
0811 12:03:24.330625 (+ 5346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:24.330631 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:24.330767 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:24.331010 (+ 243us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:24.334769 (+ 3759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:24.336389 (+ 1620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:24.340021 (+ 3632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:24.341274 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:24.341397 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:24.348377 (+ 6980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:24.348393 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:24.348399 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:24.348449 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:24.353652 (+ 5203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:24.355019 (+ 1367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:24.355027 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:24.355031 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:24.355231 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:24.356001 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:24.356006 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:24.356010 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:24.356495 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:24.356720 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:24.358094 (+ 1374us) server_negotiation.cc:299] Negotiation successful
0811 12:03:24.358371 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":734,"thread_start_us":537,"threads_started":1}
I20260811 12:03:24.360643 21611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:24.322091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:49719 (local address 127.9.221.188:34403)
0811 12:03:24.322344 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:24.322353 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:24.325260 (+ 2907us) server_negotiation.cc:407] Connection header received
0811 12:03:24.332409 (+ 7149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:24.332416 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:24.332498 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:24.332565 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:24.334876 (+ 2311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:24.336418 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:24.340021 (+ 3603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:24.341086 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:24.341205 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:24.348009 (+ 6804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:24.348027 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:24.348032 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:24.348072 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:24.353902 (+ 5830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:24.355100 (+ 1198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:24.355105 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:24.355109 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:24.355385 (+ 276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:24.356043 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:24.356050 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:24.356053 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:24.356559 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:24.356721 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:24.359579 (+ 2858us) server_negotiation.cc:299] Negotiation successful
0811 12:03:24.359912 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 12:03:24.361814 21819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:24.325540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:38537 (local address 127.9.221.129:52221)
0811 12:03:24.328478 (+ 2938us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:24.328493 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:24.330303 (+ 1810us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:24.331543 (+ 1240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:24.331551 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:24.332913 (+ 1362us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:24.334650 (+ 1737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:24.334670 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:24.336433 (+ 1763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:24.336441 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:24.339844 (+ 3403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:24.339860 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:24.341519 (+ 1659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:24.341528 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:24.341851 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:24.343186 (+ 1335us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:24.343211 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:24.348720 (+ 5509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:24.355154 (+ 6434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:24.355159 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:24.355162 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:24.355944 (+ 782us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:24.356433 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:24.356437 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:24.356441 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:24.356646 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:24.357495 (+ 849us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:24.357500 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:24.357696 (+ 196us) client_negotiation.cc:769] Sending connection context
0811 12:03:24.359112 (+ 1416us) client_negotiation.cc:240] Negotiation successful
0811 12:03:24.359432 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2779,"mutex_wait_us":390,"spinlock_wait_cycles":768,"thread_start_us":2568,"threads_started":1}
I20260811 12:03:24.361992 21815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:24.323117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:34403 (local address 127.9.221.129:49719)
0811 12:03:24.325010 (+ 1893us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:24.325028 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:24.332190 (+ 7162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:24.332867 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:24.332874 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:24.333441 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:24.334705 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:24.334716 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:24.336631 (+ 1915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:24.336637 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:24.339848 (+ 3211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:24.339859 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:24.341688 (+ 1829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:24.341696 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:24.342124 (+ 428us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:24.342994 (+ 870us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:24.343077 (+ 83us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:24.347699 (+ 4622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:24.354128 (+ 6429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:24.354134 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:24.354138 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:24.354943 (+ 805us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:24.355537 (+ 594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:24.355543 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:24.355547 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:24.355884 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:24.356911 (+ 1027us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:24.356920 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:24.357565 (+ 645us) client_negotiation.cc:769] Sending connection context
0811 12:03:24.357911 (+ 346us) client_negotiation.cc:240] Negotiation successful
0811 12:03:24.358585 (+ 674us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1759,"mutex_wait_us":403,"spinlock_wait_cycles":5888,"thread_start_us":1487,"threads_started":1}
I20260811 12:03:24.363016 21814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:24.322966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:37829 (local address 127.9.221.129:41763)
0811 12:03:24.324901 (+ 1935us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:24.324954 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:24.330302 (+ 5348us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:24.331274 (+ 972us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:24.331296 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:24.332653 (+ 1357us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:24.334450 (+ 1797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:24.334498 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:24.336631 (+ 2133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:24.336637 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:24.339830 (+ 3193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:24.339847 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:24.341940 (+ 2093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:24.341948 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:24.342223 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:24.343131 (+ 908us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:24.343165 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:24.348009 (+ 4844us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:24.353856 (+ 5847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:24.353866 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:24.353884 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:24.354864 (+ 980us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:24.355391 (+ 527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:24.355396 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:24.355399 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:24.355845 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:24.356916 (+ 1071us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:24.356923 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:24.357512 (+ 589us) client_negotiation.cc:769] Sending connection context
0811 12:03:24.359377 (+ 1865us) client_negotiation.cc:240] Negotiation successful
0811 12:03:24.359681 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1559,"mutex_wait_us":361,"spinlock_wait_cycles":82816,"thread_start_us":1169,"threads_started":1}
I20260811 12:03:24.364296 21808 heartbeater.cc:347] Connected to a master server at 127.9.221.190:38537
I20260811 12:03:24.364464 21807 heartbeater.cc:347] Connected to a master server at 127.9.221.188:34403
I20260811 12:03:24.364667 21808 heartbeater.cc:464] Registering TS with master...
I20260811 12:03:24.364705 21807 heartbeater.cc:464] Registering TS with master...
I20260811 12:03:24.365489 21810 heartbeater.cc:347] Connected to a master server at 127.9.221.189:37829
I20260811 12:03:24.365916 21810 heartbeater.cc:464] Registering TS with master...
I20260811 12:03:24.366214 21807 heartbeater.cc:511] Master 127.9.221.188:34403 requested a full tablet report, sending...
I20260811 12:03:24.366408 21808 heartbeater.cc:511] Master 127.9.221.190:38537 requested a full tablet report, sending...
I20260811 12:03:24.366896 21824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:24.329481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56044 (local address 127.9.221.130:38881)
0811 12:03:24.330418 (+ 937us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:24.330429 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:24.334889 (+ 4460us) server_negotiation.cc:407] Connection header received
0811 12:03:24.334984 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:24.334992 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:24.335256 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:24.335564 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:24.338761 (+ 3197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:24.340854 (+ 2093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:24.343125 (+ 2271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:24.344656 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:24.344860 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:24.355223 (+ 10363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:24.355251 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:24.355271 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:24.355338 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:24.361699 (+ 6361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:24.362801 (+ 1102us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:24.362809 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:24.362812 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:24.362998 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:24.363677 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:24.363734 (+ 57us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:24.363758 (+ 24us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:24.364309 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:24.365822 (+ 1513us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:24.366014 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 12:03:24.366302 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":749,"thread_start_us":550,"threads_started":1}
I20260811 12:03:24.367193 21810 heartbeater.cc:511] Master 127.9.221.189:37829 requested a full tablet report, sending...
I20260811 12:03:24.368680 20807 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:03:24.369072 20326 ts_manager.cc:194] Re-registered known tserver with Master: 5d3b7729ac6448e1a27b453a8f278fbc (127.9.221.129:34021)
I20260811 12:03:24.369365 21244 ts_manager.cc:194] Registered new tserver with Master: 5d3b7729ac6448e1a27b453a8f278fbc (127.9.221.129:34021)
I20260811 12:03:24.369635 21532 ts_manager.cc:194] Registered new tserver with Master: 5d3b7729ac6448e1a27b453a8f278fbc (127.9.221.129:34021)
I20260811 12:03:24.370384 20326 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.221.129@KRBTEST.COM'} at 127.9.221.129:52221
I20260811 12:03:24.467625 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 20617
I20260811 12:03:24.492041 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.130:38881
--local_ip_for_outbound_sockets=127.9.221.130
--tserver_master_addrs=127.9.221.190:38537
--webserver_port=35777
--webserver_interface=127.9.221.130
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.130.keytab
--principal=oryx/127.9.221.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:37859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.221.190:38537,127.9.221.189:37829
--tserver_master_addrs=127.9.221.190:38537,127.9.221.189:37829,127.9.221.188:34403 with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:03:24.571908 21469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5b972c76ffa8496faf82371b3f1523f7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } peers { permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 37829 } attrs { promote: false } } peers { permanent_uuid: "923b96c3241c4a5ab57db67d4d33856b" member_type: NON_VOTER last_known_addr { host: "127.9.221.188" port: 34403 } attrs { promote: true } } }
I20260811 12:03:24.572620 21469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 [sys.catalog]: This master's current role is: LEADER
I20260811 12:03:24.574561 21828 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7: attempting to promote NON_VOTER 923b96c3241c4a5ab57db67d4d33856b to VOTER
I20260811 12:03:24.575131 21828 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } peers { permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 37829 } attrs { promote: false } } peers { permanent_uuid: "923b96c3241c4a5ab57db67d4d33856b" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 34403 } attrs { promote: false } }
I20260811 12:03:24.577039 21550 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [term 1 LEARNER]: Refusing update from remote peer 5b972c76ffa8496faf82371b3f1523f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260811 12:03:24.577714 21255 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c [term 1 FOLLOWER]: Refusing update from remote peer 5b972c76ffa8496faf82371b3f1523f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260811 12:03:24.577749 21829 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "923b96c3241c4a5ab57db67d4d33856b" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 34403 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 12:03:24.578369 21828 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 37829 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 12:03:24.581478 21829 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 923b96c3241c4a5ab57db67d4d33856b (127.9.221.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } peers { permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 37829 } attrs { promote: false } } peers { permanent_uuid: "923b96c3241c4a5ab57db67d4d33856b" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 34403 } attrs { promote: false } } }
I20260811 12:03:24.582161 21255 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 923b96c3241c4a5ab57db67d4d33856b (127.9.221.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } peers { permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 37829 } attrs { promote: false } } peers { permanent_uuid: "923b96c3241c4a5ab57db67d4d33856b" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 34403 } attrs { promote: false } } }
I20260811 12:03:24.586133 21832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5b972c76ffa8496faf82371b3f1523f7" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } peers { permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 37829 } attrs { promote: false } } peers { permanent_uuid: "923b96c3241c4a5ab57db67d4d33856b" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 34403 } attrs { promote: false } } }
I20260811 12:03:24.586552 21832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:03:24.587364 21550 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 923b96c3241c4a5ab57db67d4d33856b (127.9.221.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } peers { permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 37829 } attrs { promote: false } } peers { permanent_uuid: "923b96c3241c4a5ab57db67d4d33856b" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 34403 } attrs { promote: false } } }
I20260811 12:03:24.588245 21469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5b972c76ffa8496faf82371b3f1523f7" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } peers { permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 37829 } attrs { promote: false } } peers { permanent_uuid: "923b96c3241c4a5ab57db67d4d33856b" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 34403 } attrs { promote: false } } }
I20260811 12:03:24.587976 21829 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "5b972c76ffa8496faf82371b3f1523f7" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } peers { permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 37829 } attrs { promote: false } } peers { permanent_uuid: "923b96c3241c4a5ab57db67d4d33856b" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 34403 } attrs { promote: false } } }
I20260811 12:03:24.588845 21469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 [sys.catalog]: This master's current role is: LEADER
I20260811 12:03:24.588922 21829 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 [sys.catalog]: This master's current role is: LEADER
I20260811 12:03:24.589228 21469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5b972c76ffa8496faf82371b3f1523f7" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } peers { permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 37829 } attrs { promote: false } } peers { permanent_uuid: "923b96c3241c4a5ab57db67d4d33856b" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 34403 } attrs { promote: false } } }
I20260811 12:03:24.589608 21469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7 [sys.catalog]: This master's current role is: LEADER
I20260811 12:03:24.591357 21592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5b972c76ffa8496faf82371b3f1523f7" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } peers { permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 37829 } attrs { promote: false } } peers { permanent_uuid: "923b96c3241c4a5ab57db67d4d33856b" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 34403 } attrs { promote: false } } }
I20260811 12:03:24.591709 21592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [sys.catalog]: This master's current role is: FOLLOWER
W20260811 12:03:24.920866 21826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:03:24.921249 21826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:03:24.921445 21826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:03:24.929519 21826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:03:24.929680 21826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:03:24.929867 21826 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.130
I20260811 12:03:24.940158 21826 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:37859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.130.keytab
--principal=oryx/127.9.221.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.130:38881
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.221.130
--webserver_port=35777
--tserver_master_addrs=127.9.221.190:38537,127.9.221.189:37829,127.9.221.188:34403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.130
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:03:24.941654 21826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:03:24.943131 21826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:03:24.957633 21839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:24.957649 21840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:24.957916 21842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:03:24.959342 21826 server_base.cc:1034] running on GCE node
I20260811 12:03:24.960078 21826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:03:24.961083 21826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:03:24.962330 21826 hybrid_clock.cc:648] HybridClock initialized: now 1786449804962250 us; error 83 us; skew 500 ppm
I20260811 12:03:24.962664 21826 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:03:24 dist-test-slave-mjjr krb5kdc[20261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449804, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.221.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:03:24.966058 21826 init.cc:377] Logged in from keytab as oryx/127.9.221.130@KRBTEST.COM (short username oryx)
I20260811 12:03:24.967633 21826 webserver.cc:460] Webserver started at http://127.9.221.130:35777/ using document root <none> and password file <none>
I20260811 12:03:24.968434 21826 fs_manager.cc:359] Metadata directory not provided
I20260811 12:03:24.968616 21826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:03:24.974370 21826 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 12:03:24.977205 21856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:24.978648 21826 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 12:03:24.978840 21826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-1/wal
uuid: "eae73ca664404c1e9ccc0414bc2daa03"
format_stamp: "Formatted at 2026-08-11 12:03:21 on dist-test-slave-mjjr"
server_key: "9e5424694b4035e367dc4491bbcb8feb"
server_key_iv: "5d979aed036cd22e1a959155559b3862"
server_key_version: "encryptionkey@0"
I20260811 12:03:24.979869 21826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:03:25.002324 21826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:03:25.006906 21826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:03:25.007220 21826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:03:25.008324 21826 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:03:25.009812 21826 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:03:25.009924 21826 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:25.010051 21826 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:03:25.010152 21826 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:25.045023 21826 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.130:38881
I20260811 12:03:25.045089 22017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.130:38881 every 8 connection(s)
I20260811 12:03:25.046579 21826 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
I20260811 12:03:25.049418 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 21826
I20260811 12:03:25.049724 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 20822
Aug 11 12:03:25 dist-test-slave-mjjr krb5kdc[20261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449804, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.221.130@KRBTEST.COM for oryx/127.9.221.189@KRBTEST.COM
Aug 11 12:03:25 dist-test-slave-mjjr krb5kdc[20261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449804, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.221.130@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM
I20260811 12:03:25.078042 22027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:25.050200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:37829 (local address 127.9.221.130:42163)
0811 12:03:25.051190 (+ 990us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:25.051240 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:25.054283 (+ 3043us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:25.057137 (+ 2854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:25.057145 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:25.057663 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:25.058219 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:25.058245 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.059666 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.059675 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:25.062334 (+ 2659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:25.062355 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.063810 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.063820 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:25.064210 (+ 390us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:25.065049 (+ 839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:25.065107 (+ 58us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:25.068897 (+ 3790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:25.073129 (+ 4232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:25.073139 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:25.073159 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:25.073869 (+ 710us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:25.074413 (+ 544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:25.074419 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:25.074422 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:25.074696 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:25.075345 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:25.075352 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:25.075817 (+ 465us) client_negotiation.cc:769] Sending connection context
0811 12:03:25.076143 (+ 326us) client_negotiation.cc:240] Negotiation successful
0811 12:03:25.076619 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":652,"thread_start_us":3228,"threads_started":1}
I20260811 12:03:25.079139 22024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:25.049806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:42163 (local address 127.9.221.189:37829)
0811 12:03:25.056597 (+ 6791us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:25.056602 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:25.056618 (+ 16us) server_negotiation.cc:407] Connection header received
0811 12:03:25.056670 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:25.056674 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:25.056749 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:25.056880 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:25.058414 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.059408 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.062588 (+ 3180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.063458 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.064679 (+ 1221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:25.069144 (+ 4465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:25.069151 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:25.069155 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:25.069201 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:25.072976 (+ 3775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:25.074085 (+ 1109us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:25.074090 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:25.074093 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:25.074269 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:25.074845 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:25.074852 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:25.074856 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:25.075198 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:25.078391 (+ 3193us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:25.078554 (+ 163us) server_negotiation.cc:299] Negotiation successful
0811 12:03:25.078792 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6700,"thread_start_us":6566,"threads_started":1}
I20260811 12:03:25.080507 22021 heartbeater.cc:347] Connected to a master server at 127.9.221.189:37829
I20260811 12:03:25.080854 22021 heartbeater.cc:464] Registering TS with master...
I20260811 12:03:25.081764 22021 heartbeater.cc:511] Master 127.9.221.189:37829 requested a full tablet report, sending...
Aug 11 12:03:25 dist-test-slave-mjjr krb5kdc[20261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449804, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.221.130@KRBTEST.COM for oryx/127.9.221.188@KRBTEST.COM
I20260811 12:03:25.083825 21244 ts_manager.cc:194] Registered new tserver with Master: eae73ca664404c1e9ccc0414bc2daa03 (127.9.221.130:38881)
Aug 11 12:03:25 dist-test-slave-mjjr krb5kdc[20261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.221.131@KRBTEST.COM
I20260811 12:03:25.085988 22030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:25.051684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:38537 (local address 127.9.221.130:54565)
0811 12:03:25.058864 (+ 7180us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:25.058883 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:25.059026 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:25.066045 (+ 7019us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:25.066052 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:25.066801 (+ 749us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:25.067190 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:25.067204 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.069169 (+ 1965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.069177 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:25.070995 (+ 1818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:25.071012 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.072167 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.072174 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:25.072343 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:25.072931 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:25.072952 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:25.075902 (+ 2950us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:25.082883 (+ 6981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:25.082891 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:25.082898 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:25.083449 (+ 551us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:25.083900 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:25.083905 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:25.083908 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:25.084075 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:25.084992 (+ 917us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:25.084996 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:25.085159 (+ 163us) client_negotiation.cc:769] Sending connection context
0811 12:03:25.085307 (+ 148us) client_negotiation.cc:240] Negotiation successful
0811 12:03:25.085586 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7036,"thread_start_us":6848,"threads_started":1}
I20260811 12:03:25.086017 22023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:25.049470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:54565 (local address 127.9.221.190:38537)
0811 12:03:25.065532 (+ 16062us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:25.065541 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:25.065564 (+ 23us) server_negotiation.cc:407] Connection header received
0811 12:03:25.065637 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:25.065645 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:25.065743 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:25.067404 (+ 1661us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:25.067584 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.068941 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.071140 (+ 2199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.071993 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.072146 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:25.077559 (+ 5413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:25.077565 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:25.077570 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:25.077606 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:25.082277 (+ 4671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:25.083567 (+ 1290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:25.083574 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:25.083580 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:25.083793 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:25.084430 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:25.084483 (+ 53us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:25.084487 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:25.084888 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:25.085020 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:25.085421 (+ 401us) server_negotiation.cc:299] Negotiation successful
0811 12:03:25.085639 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15894,"thread_start_us":16362,"threads_started":1}
I20260811 12:03:25.087239 22020 heartbeater.cc:347] Connected to a master server at 127.9.221.190:38537
I20260811 12:03:25.087435 22020 heartbeater.cc:464] Registering TS with master...
I20260811 12:03:25.087944 22020 heartbeater.cc:511] Master 127.9.221.190:38537 requested a full tablet report, sending...
I20260811 12:03:25.089210 20326 ts_manager.cc:194] Re-registered known tserver with Master: eae73ca664404c1e9ccc0414bc2daa03 (127.9.221.130:38881)
I20260811 12:03:25.090452 20326 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.221.130@KRBTEST.COM'} at 127.9.221.130:54565
I20260811 12:03:25.092653 22032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:25.059042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:34403 (local address 127.9.221.130:33463)
0811 12:03:25.072641 (+ 13599us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:25.072659 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:25.073666 (+ 1007us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:25.074272 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:25.074276 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:25.074896 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:25.075241 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:25.075259 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.076566 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.076572 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:25.077668 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:25.077681 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.079226 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.079244 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:25.079843 (+ 599us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:25.080568 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:25.080593 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:25.084622 (+ 4029us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:25.089320 (+ 4698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:25.089328 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:25.089334 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:25.089861 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:25.090287 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:25.090292 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:25.090295 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:25.090432 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:25.091484 (+ 1052us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:25.091488 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:25.091675 (+ 187us) client_negotiation.cc:769] Sending connection context
0811 12:03:25.091847 (+ 172us) client_negotiation.cc:240] Negotiation successful
0811 12:03:25.092229 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12937,"mutex_wait_us":83,"thread_start_us":12779,"threads_started":1}
I20260811 12:03:25.092691 22033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:25.059259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:33463 (local address 127.9.221.188:34403)
0811 12:03:25.067374 (+ 8115us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:25.067381 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:25.072788 (+ 5407us) server_negotiation.cc:407] Connection header received
0811 12:03:25.073849 (+ 1061us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:25.073855 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:25.073939 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:25.074046 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:25.075449 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.076434 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.077836 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.078812 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.078929 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:25.084884 (+ 5955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:25.084890 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:25.084895 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:25.084935 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:25.089174 (+ 4239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:25.090014 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:25.090022 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:25.090028 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:25.090180 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:25.090551 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:25.090556 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:25.090559 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:25.091260 (+ 701us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:25.091404 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:25.091991 (+ 587us) server_negotiation.cc:299] Negotiation successful
0811 12:03:25.092263 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7952,"thread_start_us":8978,"threads_started":1}
I20260811 12:03:25.093825 22018 heartbeater.cc:347] Connected to a master server at 127.9.221.188:34403
I20260811 12:03:25.093958 22018 heartbeater.cc:464] Registering TS with master...
I20260811 12:03:25.094061 22029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:25.051309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40106 (local address 127.9.221.131:39233)
0811 12:03:25.056212 (+ 4903us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:25.056224 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:25.069784 (+ 13560us) server_negotiation.cc:407] Connection header received
0811 12:03:25.070056 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:25.070070 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:25.070384 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:25.070529 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:25.074081 (+ 3552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.075602 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.077458 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.078302 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.078418 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:25.086073 (+ 7655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:25.086092 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:25.086126 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:25.086170 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:25.090184 (+ 4014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:25.091065 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:25.091073 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:25.091077 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:25.091219 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:25.091643 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:25.091648 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:25.091709 (+ 61us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:25.092420 (+ 711us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:25.092719 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:25.093323 (+ 604us) server_negotiation.cc:299] Negotiation successful
0811 12:03:25.093596 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4697,"thread_start_us":4526,"threads_started":1}
I20260811 12:03:25.095082 22018 heartbeater.cc:511] Master 127.9.221.188:34403 requested a full tablet report, sending...
I20260811 12:03:25.095247 21004 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:03:25.096367 21532 ts_manager.cc:194] Registered new tserver with Master: eae73ca664404c1e9ccc0414bc2daa03 (127.9.221.130:38881)
I20260811 12:03:25.179788 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 20822
I20260811 12:03:25.203344 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.131:39233
--local_ip_for_outbound_sockets=127.9.221.131
--tserver_master_addrs=127.9.221.190:38537
--webserver_port=43067
--webserver_interface=127.9.221.131
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.131.keytab
--principal=oryx/127.9.221.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:37859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.221.190:38537,127.9.221.189:37829
--tserver_master_addrs=127.9.221.190:38537,127.9.221.189:37829,127.9.221.188:34403 with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:03:25.372937 21808 heartbeater.cc:503] Master 127.9.221.190:38537 was elected leader, sending a full tablet report...
W20260811 12:03:25.547456 22035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:03:25.547843 22035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:03:25.548022 22035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:03:25.555809 22035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:03:25.555944 22035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:03:25.556149 22035 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.131
I20260811 12:03:25.566290 22035 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:37859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.131.keytab
--principal=oryx/127.9.221.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.131:39233
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.221.131
--webserver_port=43067
--tserver_master_addrs=127.9.221.190:38537,127.9.221.189:37829,127.9.221.188:34403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.131
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:03:25.568289 22035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:03:25.569872 22035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:03:25.584286 22042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:25.584256 22043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:25.584206 22046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:03:25.585143 22035 server_base.cc:1034] running on GCE node
I20260811 12:03:25.585793 22035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:03:25.587028 22035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:03:25.588269 22035 hybrid_clock.cc:648] HybridClock initialized: now 1786449805588219 us; error 67 us; skew 500 ppm
I20260811 12:03:25.588615 22035 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:03:25 dist-test-slave-mjjr krb5kdc[20261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449805, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.221.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:03:25.591871 22035 init.cc:377] Logged in from keytab as oryx/127.9.221.131@KRBTEST.COM (short username oryx)
I20260811 12:03:25.593433 22035 webserver.cc:460] Webserver started at http://127.9.221.131:43067/ using document root <none> and password file <none>
I20260811 12:03:25.594194 22035 fs_manager.cc:359] Metadata directory not provided
I20260811 12:03:25.594358 22035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:03:25.600081 22035 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 12:03:25.602936 22056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:25.604475 22035 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:03:25.604616 22035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-2/wal
uuid: "3d4a8b75d0574794b67bb8e235e307b2"
format_stamp: "Formatted at 2026-08-11 12:03:21 on dist-test-slave-mjjr"
server_key: "6a9bf2ea925d8e1edf23528384958ad6"
server_key_iv: "8de5cd9a3a166de349d0a9b51a0fff00"
server_key_version: "encryptionkey@0"
I20260811 12:03:25.605569 22035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:03:25.618340 22035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:03:25.622674 22035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:03:25.623008 22035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:03:25.624032 22035 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:03:25.625393 22035 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:03:25.625530 22035 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:25.625627 22035 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:03:25.625721 22035 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:25.659862 22035 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.131:39233
I20260811 12:03:25.659951 22232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.131:39233 every 8 connection(s)
I20260811 12:03:25.661300 22035 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
I20260811 12:03:25.667361 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 22035
Aug 11 12:03:25 dist-test-slave-mjjr krb5kdc[20261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449805, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.221.131@KRBTEST.COM for oryx/127.9.221.189@KRBTEST.COM
Aug 11 12:03:25 dist-test-slave-mjjr krb5kdc[20261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449805, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.221.131@KRBTEST.COM for oryx/127.9.221.188@KRBTEST.COM
Aug 11 12:03:25 dist-test-slave-mjjr krb5kdc[20261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449805, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.221.131@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM
I20260811 12:03:25.718117 22242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:25.664298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:53999 (local address 127.9.221.188:34403)
0811 12:03:25.677017 (+ 12719us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:25.677027 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:25.681205 (+ 4178us) server_negotiation.cc:407] Connection header received
0811 12:03:25.682304 (+ 1099us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:25.682314 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:25.682432 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:25.682546 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:25.687051 (+ 4505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.688190 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.691951 (+ 3761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.692644 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.692778 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:25.698954 (+ 6176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:25.698961 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:25.698967 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:25.699011 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:25.706742 (+ 7731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:25.708524 (+ 1782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:25.708529 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:25.708533 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:25.708982 (+ 449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:25.716047 (+ 7065us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:25.716054 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:25.716058 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:25.716493 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:25.716645 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:25.717325 (+ 680us) server_negotiation.cc:299] Negotiation successful
0811 12:03:25.717596 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12543,"thread_start_us":13172,"threads_started":1}
I20260811 12:03:25.718783 22261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:25.690086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55606 (local address 127.9.221.188:34403)
0811 12:03:25.694174 (+ 4088us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:25.694182 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:25.701026 (+ 6844us) server_negotiation.cc:407] Connection header received
0811 12:03:25.701193 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:25.701198 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:25.701273 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:25.701383 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:25.704007 (+ 2624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.705242 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.707193 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.707870 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.707967 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:25.711588 (+ 3621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:25.711596 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:25.711601 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:25.711662 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:25.715289 (+ 3627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:25.716215 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:25.716220 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:25.716224 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:25.716400 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:25.716870 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:25.716879 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:25.716884 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:25.717233 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:25.717507 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:25.718171 (+ 664us) server_negotiation.cc:299] Negotiation successful
0811 12:03:25.718386 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3933,"thread_start_us":3216,"threads_started":1}
I20260811 12:03:25.721323 22258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:25.680323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:35603 (local address 127.9.221.190:38537)
0811 12:03:25.688255 (+ 7932us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:25.688264 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:25.688288 (+ 24us) server_negotiation.cc:407] Connection header received
0811 12:03:25.688364 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:25.688372 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:25.688482 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:25.688618 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:25.694686 (+ 6068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.695842 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.702087 (+ 6245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.702965 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.703091 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:25.708751 (+ 5660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:25.708760 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:25.708767 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:25.708807 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:25.713342 (+ 4535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:25.718429 (+ 5087us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:25.718437 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:25.718442 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:25.718565 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:25.718997 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:25.719002 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:25.719006 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:25.719390 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:25.719545 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:25.720612 (+ 1067us) server_negotiation.cc:299] Negotiation successful
0811 12:03:25.720847 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7754,"thread_start_us":13520,"threads_started":1}
I20260811 12:03:25.721992 22256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:25.680165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:38537 (local address 127.9.221.131:35603)
0811 12:03:25.685592 (+ 5427us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:25.685632 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:25.685790 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:25.690374 (+ 4584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:25.690380 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:25.690999 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:25.691352 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:25.691367 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.696555 (+ 5188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.696563 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:25.698665 (+ 2102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:25.698676 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.703410 (+ 4734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.703417 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:25.703703 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:25.704347 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:25.705420 (+ 1073us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:25.708527 (+ 3107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:25.716865 (+ 8338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:25.716871 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:25.716874 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:25.717404 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:25.718684 (+ 1280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:25.718689 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:25.718693 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:25.718873 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:25.720112 (+ 1239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:25.720117 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:25.720307 (+ 190us) client_negotiation.cc:769] Sending connection context
0811 12:03:25.720471 (+ 164us) client_negotiation.cc:240] Negotiation successful
0811 12:03:25.720759 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5256,"thread_start_us":5733,"threads_started":1}
I20260811 12:03:25.722111 22254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:25.677354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43712 (local address 127.9.221.189:37829)
0811 12:03:25.686486 (+ 9132us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:25.686493 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:25.689492 (+ 2999us) server_negotiation.cc:407] Connection header received
0811 12:03:25.694166 (+ 4674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:25.694173 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:25.694270 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:25.694410 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:25.704497 (+ 10087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.705685 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.710482 (+ 4797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.711186 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.711284 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:25.714837 (+ 3553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:25.714844 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:25.714849 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:25.714890 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:25.718440 (+ 3550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:25.719852 (+ 1412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:25.719860 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:25.719863 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:25.720035 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:25.720475 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:25.720480 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:25.720483 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:25.720812 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:25.721032 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:25.721662 (+ 630us) server_negotiation.cc:299] Negotiation successful
0811 12:03:25.721817 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9005,"thread_start_us":8898,"threads_started":1}
I20260811 12:03:25.722566 22243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:25.664454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:35275 (local address 127.9.221.189:37829)
0811 12:03:25.677433 (+ 12979us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:25.677442 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:25.677469 (+ 27us) server_negotiation.cc:407] Connection header received
0811 12:03:25.679036 (+ 1567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:25.679045 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:25.679155 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:25.679305 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:25.688053 (+ 8748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.689172 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.691410 (+ 2238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.692160 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.692279 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:25.699731 (+ 7452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:25.699740 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:25.699747 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:25.699791 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:25.706002 (+ 6211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:25.709171 (+ 3169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:25.709178 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:25.709183 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:25.709401 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:25.714371 (+ 4970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:25.714379 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:25.714383 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:25.714791 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:25.714987 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:25.721761 (+ 6774us) server_negotiation.cc:299] Negotiation successful
0811 12:03:25.722069 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12828,"thread_start_us":12709,"threads_started":1}
I20260811 12:03:25.723541 22235 heartbeater.cc:347] Connected to a master server at 127.9.221.190:38537
I20260811 12:03:25.723628 22246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:25.665032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:37829 (local address 127.9.221.131:35275)
0811 12:03:25.675510 (+ 10478us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:25.675567 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:25.678600 (+ 3033us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:25.679876 (+ 1276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:25.679892 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:25.687526 (+ 7634us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:25.687846 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:25.687859 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.689400 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.689406 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:25.691195 (+ 1789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:25.691225 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.692649 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.692657 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:25.692981 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:25.693875 (+ 894us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:25.693908 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:25.699450 (+ 5542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:25.706934 (+ 7484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:25.706939 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:25.706954 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:25.708182 (+ 1228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:25.714023 (+ 5841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:25.714028 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:25.714032 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:25.714218 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:25.715103 (+ 885us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:25.715113 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:25.715589 (+ 476us) client_negotiation.cc:769] Sending connection context
0811 12:03:25.722800 (+ 7211us) client_negotiation.cc:240] Negotiation successful
0811 12:03:25.723113 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10103,"mutex_wait_us":1,"spinlock_wait_cycles":3968,"thread_start_us":15534,"threads_started":1}
I20260811 12:03:25.723961 22235 heartbeater.cc:464] Registering TS with master...
I20260811 12:03:25.724897 22257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:25.680776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37946 (local address 127.9.221.190:38537)
0811 12:03:25.693780 (+ 13004us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:25.693789 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:25.696677 (+ 2888us) server_negotiation.cc:407] Connection header received
0811 12:03:25.696881 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:25.696888 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:25.696973 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:25.697100 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:25.700591 (+ 3491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.701836 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.704436 (+ 2600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.705125 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.705274 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:25.712688 (+ 7414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:25.712697 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:25.712702 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:25.712746 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:25.716727 (+ 3981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:25.718358 (+ 1631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:25.718363 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:25.718366 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:25.718526 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:25.719383 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:25.719388 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:25.719391 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:25.719782 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:25.720029 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:25.724151 (+ 4122us) server_negotiation.cc:299] Negotiation successful
0811 12:03:25.724360 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12856,"thread_start_us":7226,"threads_started":1}
I20260811 12:03:25.724913 22235 heartbeater.cc:511] Master 127.9.221.190:38537 requested a full tablet report, sending...
I20260811 12:03:25.725133 22245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:25.665032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:34403 (local address 127.9.221.131:53999)
0811 12:03:25.680949 (+ 15917us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:25.681069 (+ 120us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:25.681925 (+ 856us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:25.682873 (+ 948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:25.682882 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:25.684130 (+ 1248us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:25.686856 (+ 2726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:25.686876 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.688393 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.688399 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:25.691761 (+ 3362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:25.691775 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.693104 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.693123 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:25.693316 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:25.695565 (+ 2249us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:25.695591 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:25.698675 (+ 3084us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:25.706928 (+ 8253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:25.706937 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:25.706954 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:25.708255 (+ 1301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:25.713556 (+ 5301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:25.713564 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:25.713569 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:25.713862 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:25.716749 (+ 2887us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:25.716755 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:25.716932 (+ 177us) client_negotiation.cc:769] Sending connection context
0811 12:03:25.718686 (+ 1754us) client_negotiation.cc:240] Negotiation successful
0811 12:03:25.720024 (+ 1338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15766,"mutex_wait_us":1226,"spinlock_wait_cycles":78336,"thread_start_us":13884,"threads_started":1}
I20260811 12:03:25.726099 22234 heartbeater.cc:347] Connected to a master server at 127.9.221.188:34403
I20260811 12:03:25.726250 22234 heartbeater.cc:464] Registering TS with master...
I20260811 12:03:25.726339 22237 heartbeater.cc:347] Connected to a master server at 127.9.221.189:37829
I20260811 12:03:25.726480 22237 heartbeater.cc:464] Registering TS with master...
I20260811 12:03:25.726588 20326 ts_manager.cc:194] Re-registered known tserver with Master: 3d4a8b75d0574794b67bb8e235e307b2 (127.9.221.131:39233)
I20260811 12:03:25.726864 22234 heartbeater.cc:511] Master 127.9.221.188:34403 requested a full tablet report, sending...
I20260811 12:03:25.727617 22237 heartbeater.cc:511] Master 127.9.221.189:37829 requested a full tablet report, sending...
I20260811 12:03:25.727766 20326 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.221.131@KRBTEST.COM'} at 127.9.221.131:35603
I20260811 12:03:25.728690 21244 ts_manager.cc:194] Registered new tserver with Master: 3d4a8b75d0574794b67bb8e235e307b2 (127.9.221.131:39233)
I20260811 12:03:25.729110 21532 ts_manager.cc:194] Registered new tserver with Master: 3d4a8b75d0574794b67bb8e235e307b2 (127.9.221.131:39233)
I20260811 12:03:25.730868 20326 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37946:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:03:25.737803 20326 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:03:25.767448 22284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:25.758853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54576 (local address 127.9.221.130:38881)
0811 12:03:25.759372 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:25.759380 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:25.759403 (+ 23us) server_negotiation.cc:407] Connection header received
0811 12:03:25.759492 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:25.759498 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:25.760149 (+ 651us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:25.760371 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:03:25.761287 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.762537 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.765136 (+ 2599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.766355 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.766523 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:25.766746 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:25.767005 (+ 259us) server_negotiation.cc:299] Negotiation successful
0811 12:03:25.767161 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":1374,"threads_started":1}
I20260811 12:03:25.767493 22283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:25.758486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.130:38881 (local address 127.0.0.1:54576)
0811 12:03:25.758978 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:25.758994 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:25.759158 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:25.760346 (+ 1188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:25.760350 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:25.760376 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:03:25.761050 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:25.761069 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.762742 (+ 1673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.762749 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:25.764939 (+ 2190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:25.764963 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.766598 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.766605 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:25.766786 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:25.766820 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 12:03:25.766945 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 12:03:25.767075 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"mutex_wait_us":1,"thread_start_us":533,"threads_started":1}
I20260811 12:03:25.768781 22287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:25.759330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:34021 (local address 127.0.0.1:56782)
0811 12:03:25.759908 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:25.759930 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:25.760058 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:25.761199 (+ 1141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:25.761204 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:25.761229 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:03:25.761614 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:25.761627 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.763413 (+ 1786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.763419 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:25.765395 (+ 1976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:25.765409 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.767972 (+ 2563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.767980 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:25.768180 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:25.768262 (+ 82us) client_negotiation.cc:769] Sending connection context
0811 12:03:25.768333 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 12:03:25.768431 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"mutex_wait_us":1,"thread_start_us":778,"threads_started":1}
I20260811 12:03:25.769691 22288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:25.759924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56782 (local address 127.9.221.129:34021)
0811 12:03:25.760446 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:25.760455 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:25.760483 (+ 28us) server_negotiation.cc:407] Connection header received
0811 12:03:25.760591 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:25.760598 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:25.760831 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:25.761042 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:03:25.761974 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.763245 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.765560 (+ 2315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.767359 (+ 1799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.767485 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:25.769188 (+ 1703us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:25.769249 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 12:03:25.769408 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":1231,"threads_started":1}
I20260811 12:03:25.770668 22290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:25.760670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.131:39233 (local address 127.0.0.1:43074)
0811 12:03:25.761249 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:25.761272 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:25.761386 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:25.763290 (+ 1904us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:25.763298 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:25.763338 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:03:25.764000 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:25.764013 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.766019 (+ 2006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.766025 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:25.768206 (+ 2181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:25.768219 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.769730 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.769736 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:25.770061 (+ 325us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:25.770084 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 12:03:25.770191 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 12:03:25.770313 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"mutex_wait_us":2,"thread_start_us":656,"threads_started":1}
I20260811 12:03:25.770740 22292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:25.761023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43074 (local address 127.9.221.131:39233)
0811 12:03:25.762723 (+ 1700us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:25.762732 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:25.762761 (+ 29us) server_negotiation.cc:407] Connection header received
0811 12:03:25.762858 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:25.762867 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:25.763093 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:25.763277 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:03:25.764234 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.765878 (+ 1644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.768368 (+ 2490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.769513 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.769664 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:25.769867 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:25.770245 (+ 378us) server_negotiation.cc:299] Negotiation successful
0811 12:03:25.770404 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1502,"thread_start_us":1296,"threads_started":1}
I20260811 12:03:25.772244 21925 tablet_service.cc:1467] Processing CreateTablet for tablet d25d8c3eff054dbd8555e54403a5fc90 (DEFAULT_TABLE table=test-table [id=52c7f788b52d49acb67a4cd169d4b49a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:03:25.773104 21717 tablet_service.cc:1467] Processing CreateTablet for tablet d25d8c3eff054dbd8555e54403a5fc90 (DEFAULT_TABLE table=test-table [id=52c7f788b52d49acb67a4cd169d4b49a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:03:25.773773 21925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d25d8c3eff054dbd8555e54403a5fc90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:03:25.774406 21717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d25d8c3eff054dbd8555e54403a5fc90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:03:25.777684 22125 tablet_service.cc:1467] Processing CreateTablet for tablet d25d8c3eff054dbd8555e54403a5fc90 (DEFAULT_TABLE table=test-table [id=52c7f788b52d49acb67a4cd169d4b49a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:03:25.778851 22125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d25d8c3eff054dbd8555e54403a5fc90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:03:25.789543 22293 tablet_bootstrap.cc:492] T d25d8c3eff054dbd8555e54403a5fc90 P 3d4a8b75d0574794b67bb8e235e307b2: Bootstrap starting.
I20260811 12:03:25.792305 22295 tablet_bootstrap.cc:492] T d25d8c3eff054dbd8555e54403a5fc90 P eae73ca664404c1e9ccc0414bc2daa03: Bootstrap starting.
I20260811 12:03:25.793282 22293 tablet_bootstrap.cc:654] T d25d8c3eff054dbd8555e54403a5fc90 P 3d4a8b75d0574794b67bb8e235e307b2: Neither blocks nor log segments found. Creating new log.
I20260811 12:03:25.794692 22293 log.cc:824] T d25d8c3eff054dbd8555e54403a5fc90 P 3d4a8b75d0574794b67bb8e235e307b2: Log is configured to *not* fsync() on all Append() calls
I20260811 12:03:25.795326 22294 tablet_bootstrap.cc:492] T d25d8c3eff054dbd8555e54403a5fc90 P 5d3b7729ac6448e1a27b453a8f278fbc: Bootstrap starting.
I20260811 12:03:25.795895 22295 tablet_bootstrap.cc:654] T d25d8c3eff054dbd8555e54403a5fc90 P eae73ca664404c1e9ccc0414bc2daa03: Neither blocks nor log segments found. Creating new log.
I20260811 12:03:25.797382 22295 log.cc:824] T d25d8c3eff054dbd8555e54403a5fc90 P eae73ca664404c1e9ccc0414bc2daa03: Log is configured to *not* fsync() on all Append() calls
I20260811 12:03:25.798142 22294 tablet_bootstrap.cc:654] T d25d8c3eff054dbd8555e54403a5fc90 P 5d3b7729ac6448e1a27b453a8f278fbc: Neither blocks nor log segments found. Creating new log.
I20260811 12:03:25.799296 22294 log.cc:824] T d25d8c3eff054dbd8555e54403a5fc90 P 5d3b7729ac6448e1a27b453a8f278fbc: Log is configured to *not* fsync() on all Append() calls
I20260811 12:03:25.801595 22295 tablet_bootstrap.cc:492] T d25d8c3eff054dbd8555e54403a5fc90 P eae73ca664404c1e9ccc0414bc2daa03: No bootstrap required, opened a new log
I20260811 12:03:25.801717 22294 tablet_bootstrap.cc:492] T d25d8c3eff054dbd8555e54403a5fc90 P 5d3b7729ac6448e1a27b453a8f278fbc: No bootstrap required, opened a new log
I20260811 12:03:25.801896 22295 ts_tablet_manager.cc:1397] T d25d8c3eff054dbd8555e54403a5fc90 P eae73ca664404c1e9ccc0414bc2daa03: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260811 12:03:25.802018 22294 ts_tablet_manager.cc:1397] T d25d8c3eff054dbd8555e54403a5fc90 P 5d3b7729ac6448e1a27b453a8f278fbc: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260811 12:03:25.804617 22293 tablet_bootstrap.cc:492] T d25d8c3eff054dbd8555e54403a5fc90 P 3d4a8b75d0574794b67bb8e235e307b2: No bootstrap required, opened a new log
I20260811 12:03:25.804868 22293 ts_tablet_manager.cc:1397] T d25d8c3eff054dbd8555e54403a5fc90 P 3d4a8b75d0574794b67bb8e235e307b2: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.008s
I20260811 12:03:25.813476 22294 raft_consensus.cc:348] T d25d8c3eff054dbd8555e54403a5fc90 P 5d3b7729ac6448e1a27b453a8f278fbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eae73ca664404c1e9ccc0414bc2daa03" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38881 } } peers { permanent_uuid: "3d4a8b75d0574794b67bb8e235e307b2" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39233 } } peers { permanent_uuid: "5d3b7729ac6448e1a27b453a8f278fbc" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 34021 } }
I20260811 12:03:25.814092 22294 raft_consensus.cc:374] T d25d8c3eff054dbd8555e54403a5fc90 P 5d3b7729ac6448e1a27b453a8f278fbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:03:25.814221 22294 raft_consensus.cc:729] T d25d8c3eff054dbd8555e54403a5fc90 P 5d3b7729ac6448e1a27b453a8f278fbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d3b7729ac6448e1a27b453a8f278fbc, State: Initialized, Role: FOLLOWER
I20260811 12:03:25.813977 22295 raft_consensus.cc:348] T d25d8c3eff054dbd8555e54403a5fc90 P eae73ca664404c1e9ccc0414bc2daa03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eae73ca664404c1e9ccc0414bc2daa03" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38881 } } peers { permanent_uuid: "3d4a8b75d0574794b67bb8e235e307b2" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39233 } } peers { permanent_uuid: "5d3b7729ac6448e1a27b453a8f278fbc" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 34021 } }
I20260811 12:03:25.814394 22295 raft_consensus.cc:374] T d25d8c3eff054dbd8555e54403a5fc90 P eae73ca664404c1e9ccc0414bc2daa03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:03:25.814489 22295 raft_consensus.cc:729] T d25d8c3eff054dbd8555e54403a5fc90 P eae73ca664404c1e9ccc0414bc2daa03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eae73ca664404c1e9ccc0414bc2daa03, State: Initialized, Role: FOLLOWER
I20260811 12:03:25.814961 22294 consensus_queue.cc:260] T d25d8c3eff054dbd8555e54403a5fc90 P 5d3b7729ac6448e1a27b453a8f278fbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eae73ca664404c1e9ccc0414bc2daa03" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38881 } } peers { permanent_uuid: "3d4a8b75d0574794b67bb8e235e307b2" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39233 } } peers { permanent_uuid: "5d3b7729ac6448e1a27b453a8f278fbc" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 34021 } }
I20260811 12:03:25.815169 22295 consensus_queue.cc:260] T d25d8c3eff054dbd8555e54403a5fc90 P eae73ca664404c1e9ccc0414bc2daa03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eae73ca664404c1e9ccc0414bc2daa03" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38881 } } peers { permanent_uuid: "3d4a8b75d0574794b67bb8e235e307b2" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39233 } } peers { permanent_uuid: "5d3b7729ac6448e1a27b453a8f278fbc" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 34021 } }
I20260811 12:03:25.821099 22293 raft_consensus.cc:348] T d25d8c3eff054dbd8555e54403a5fc90 P 3d4a8b75d0574794b67bb8e235e307b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eae73ca664404c1e9ccc0414bc2daa03" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38881 } } peers { permanent_uuid: "3d4a8b75d0574794b67bb8e235e307b2" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39233 } } peers { permanent_uuid: "5d3b7729ac6448e1a27b453a8f278fbc" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 34021 } }
I20260811 12:03:25.821758 22293 raft_consensus.cc:374] T d25d8c3eff054dbd8555e54403a5fc90 P 3d4a8b75d0574794b67bb8e235e307b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:03:25.821905 22293 raft_consensus.cc:729] T d25d8c3eff054dbd8555e54403a5fc90 P 3d4a8b75d0574794b67bb8e235e307b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d4a8b75d0574794b67bb8e235e307b2, State: Initialized, Role: FOLLOWER
I20260811 12:03:25.822685 22293 consensus_queue.cc:260] T d25d8c3eff054dbd8555e54403a5fc90 P 3d4a8b75d0574794b67bb8e235e307b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eae73ca664404c1e9ccc0414bc2daa03" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38881 } } peers { permanent_uuid: "3d4a8b75d0574794b67bb8e235e307b2" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39233 } } peers { permanent_uuid: "5d3b7729ac6448e1a27b453a8f278fbc" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 34021 } }
I20260811 12:03:25.824115 22294 ts_tablet_manager.cc:1428] T d25d8c3eff054dbd8555e54403a5fc90 P 5d3b7729ac6448e1a27b453a8f278fbc: Time spent starting tablet: real 0.022s user 0.015s sys 0.007s
I20260811 12:03:25.824134 22020 heartbeater.cc:503] Master 127.9.221.190:38537 was elected leader, sending a full tablet report...
I20260811 12:03:25.824793 22295 ts_tablet_manager.cc:1428] T d25d8c3eff054dbd8555e54403a5fc90 P eae73ca664404c1e9ccc0414bc2daa03: Time spent starting tablet: real 0.023s user 0.010s sys 0.012s
W20260811 12:03:25.826143 21811 tablet.cc:2369] T d25d8c3eff054dbd8555e54403a5fc90 P 5d3b7729ac6448e1a27b453a8f278fbc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:03:25.828099 22293 ts_tablet_manager.cc:1428] T d25d8c3eff054dbd8555e54403a5fc90 P 3d4a8b75d0574794b67bb8e235e307b2: Time spent starting tablet: real 0.023s user 0.016s sys 0.003s
I20260811 12:03:25.829774 22235 heartbeater.cc:503] Master 127.9.221.190:38537 was elected leader, sending a full tablet report...
I20260811 12:03:25.879768 22301 raft_consensus.cc:482] T d25d8c3eff054dbd8555e54403a5fc90 P eae73ca664404c1e9ccc0414bc2daa03 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:03:25.880183 22301 raft_consensus.cc:504] T d25d8c3eff054dbd8555e54403a5fc90 P eae73ca664404c1e9ccc0414bc2daa03 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eae73ca664404c1e9ccc0414bc2daa03" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38881 } } peers { permanent_uuid: "3d4a8b75d0574794b67bb8e235e307b2" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39233 } } peers { permanent_uuid: "5d3b7729ac6448e1a27b453a8f278fbc" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 34021 } }
I20260811 12:03:25.882340 22301 leader_election.cc:288] T d25d8c3eff054dbd8555e54403a5fc90 P eae73ca664404c1e9ccc0414bc2daa03 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d4a8b75d0574794b67bb8e235e307b2 (127.9.221.131:39233), 5d3b7729ac6448e1a27b453a8f278fbc (127.9.221.129:34021)
I20260811 12:03:25.894789 22292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:25.883163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:44477 (local address 127.9.221.131:39233)
0811 12:03:25.883493 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:25.883499 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:25.884846 (+ 1347us) server_negotiation.cc:407] Connection header received
0811 12:03:25.885204 (+ 358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:25.885210 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:25.885292 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:25.885393 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:03:25.887045 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.888584 (+ 1539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.892038 (+ 3454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.893530 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.893742 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:25.893822 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:25.894326 (+ 504us) server_negotiation.cc:299] Negotiation successful
0811 12:03:25.894472 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153}
I20260811 12:03:25.894879 22288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:25.882882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:49609 (local address 127.9.221.129:34021)
0811 12:03:25.883188 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:25.883195 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:25.885172 (+ 1977us) server_negotiation.cc:407] Connection header received
0811 12:03:25.885507 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:25.885513 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:25.885617 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:25.885690 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:03:25.887129 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.888153 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.892038 (+ 3885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.893554 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.893741 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:25.893822 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:25.894415 (+ 593us) server_negotiation.cc:299] Negotiation successful
0811 12:03:25.894544 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 12:03:25.895035 22305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:25.883479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:34021 (local address 127.9.221.130:49609)
0811 12:03:25.885051 (+ 1572us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:25.885066 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:25.885311 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:25.885910 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:25.885914 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:25.885938 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:03:25.886901 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:25.886912 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.888300 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.888305 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:25.891725 (+ 3420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:25.891741 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.894028 (+ 2287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.894034 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:25.894230 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:25.894274 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 12:03:25.894369 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 12:03:25.894471 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1430,"mutex_wait_us":40,"thread_start_us":1219,"threads_started":1}
I20260811 12:03:25.895056 22303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:25.882548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.131:39233 (local address 127.9.221.130:44477)
0811 12:03:25.884704 (+ 2156us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:25.884758 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:25.885017 (+ 259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:25.885402 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:25.885407 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:25.885434 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:03:25.886455 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:25.886485 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.888787 (+ 2302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.888801 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:25.891813 (+ 3012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:25.891828 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.893915 (+ 2087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.893923 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:25.894142 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:25.894179 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 12:03:25.894293 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 12:03:25.894442 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1970,"mutex_wait_us":27,"thread_start_us":1471,"threads_started":1}
I20260811 12:03:25.896225 21743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25d8c3eff054dbd8555e54403a5fc90" candidate_uuid: "eae73ca664404c1e9ccc0414bc2daa03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d3b7729ac6448e1a27b453a8f278fbc" is_pre_election: true
I20260811 12:03:25.896581 22155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25d8c3eff054dbd8555e54403a5fc90" candidate_uuid: "eae73ca664404c1e9ccc0414bc2daa03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d4a8b75d0574794b67bb8e235e307b2" is_pre_election: true
I20260811 12:03:25.896806 21743 raft_consensus.cc:2454] T d25d8c3eff054dbd8555e54403a5fc90 P 5d3b7729ac6448e1a27b453a8f278fbc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eae73ca664404c1e9ccc0414bc2daa03 in term 0.
I20260811 12:03:25.897248 22155 raft_consensus.cc:2454] T d25d8c3eff054dbd8555e54403a5fc90 P 3d4a8b75d0574794b67bb8e235e307b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eae73ca664404c1e9ccc0414bc2daa03 in term 0.
I20260811 12:03:25.897781 21859 leader_election.cc:302] T d25d8c3eff054dbd8555e54403a5fc90 P eae73ca664404c1e9ccc0414bc2daa03 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d3b7729ac6448e1a27b453a8f278fbc, eae73ca664404c1e9ccc0414bc2daa03; no voters:
I20260811 12:03:25.898355 22301 raft_consensus.cc:2781] T d25d8c3eff054dbd8555e54403a5fc90 P eae73ca664404c1e9ccc0414bc2daa03 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:03:25.898592 22301 raft_consensus.cc:482] T d25d8c3eff054dbd8555e54403a5fc90 P eae73ca664404c1e9ccc0414bc2daa03 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:03:25.898767 22301 raft_consensus.cc:3037] T d25d8c3eff054dbd8555e54403a5fc90 P eae73ca664404c1e9ccc0414bc2daa03 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:03:25.901754 22301 raft_consensus.cc:504] T d25d8c3eff054dbd8555e54403a5fc90 P eae73ca664404c1e9ccc0414bc2daa03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eae73ca664404c1e9ccc0414bc2daa03" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38881 } } peers { permanent_uuid: "3d4a8b75d0574794b67bb8e235e307b2" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39233 } } peers { permanent_uuid: "5d3b7729ac6448e1a27b453a8f278fbc" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 34021 } }
I20260811 12:03:25.902496 22301 leader_election.cc:288] T d25d8c3eff054dbd8555e54403a5fc90 P eae73ca664404c1e9ccc0414bc2daa03 [CANDIDATE]: Term 1 election: Requested vote from peers 3d4a8b75d0574794b67bb8e235e307b2 (127.9.221.131:39233), 5d3b7729ac6448e1a27b453a8f278fbc (127.9.221.129:34021)
I20260811 12:03:25.903003 21743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25d8c3eff054dbd8555e54403a5fc90" candidate_uuid: "eae73ca664404c1e9ccc0414bc2daa03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d3b7729ac6448e1a27b453a8f278fbc"
I20260811 12:03:25.903003 22155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25d8c3eff054dbd8555e54403a5fc90" candidate_uuid: "eae73ca664404c1e9ccc0414bc2daa03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d4a8b75d0574794b67bb8e235e307b2"
I20260811 12:03:25.903272 22155 raft_consensus.cc:3037] T d25d8c3eff054dbd8555e54403a5fc90 P 3d4a8b75d0574794b67bb8e235e307b2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:03:25.903291 21743 raft_consensus.cc:3037] T d25d8c3eff054dbd8555e54403a5fc90 P 5d3b7729ac6448e1a27b453a8f278fbc [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:03:25.906275 21743 raft_consensus.cc:2454] T d25d8c3eff054dbd8555e54403a5fc90 P 5d3b7729ac6448e1a27b453a8f278fbc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eae73ca664404c1e9ccc0414bc2daa03 in term 1.
I20260811 12:03:25.906476 22155 raft_consensus.cc:2454] T d25d8c3eff054dbd8555e54403a5fc90 P 3d4a8b75d0574794b67bb8e235e307b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eae73ca664404c1e9ccc0414bc2daa03 in term 1.
I20260811 12:03:25.906745 21859 leader_election.cc:302] T d25d8c3eff054dbd8555e54403a5fc90 P eae73ca664404c1e9ccc0414bc2daa03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d3b7729ac6448e1a27b453a8f278fbc, eae73ca664404c1e9ccc0414bc2daa03; no voters:
I20260811 12:03:25.907078 22301 raft_consensus.cc:2781] T d25d8c3eff054dbd8555e54403a5fc90 P eae73ca664404c1e9ccc0414bc2daa03 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:03:25.907789 22301 raft_consensus.cc:686] T d25d8c3eff054dbd8555e54403a5fc90 P eae73ca664404c1e9ccc0414bc2daa03 [term 1 LEADER]: Becoming Leader. State: Replica: eae73ca664404c1e9ccc0414bc2daa03, State: Running, Role: LEADER
I20260811 12:03:25.908399 22301 consensus_queue.cc:237] T d25d8c3eff054dbd8555e54403a5fc90 P eae73ca664404c1e9ccc0414bc2daa03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eae73ca664404c1e9ccc0414bc2daa03" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38881 } } peers { permanent_uuid: "3d4a8b75d0574794b67bb8e235e307b2" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39233 } } peers { permanent_uuid: "5d3b7729ac6448e1a27b453a8f278fbc" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 34021 } }
I20260811 12:03:25.914361 20326 catalog_manager.cc:5495] T d25d8c3eff054dbd8555e54403a5fc90 P eae73ca664404c1e9ccc0414bc2daa03 reported cstate change: term changed from 0 to 1, leader changed from <none> to eae73ca664404c1e9ccc0414bc2daa03 (127.9.221.130). New cstate: current_term: 1 leader_uuid: "eae73ca664404c1e9ccc0414bc2daa03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eae73ca664404c1e9ccc0414bc2daa03" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d4a8b75d0574794b67bb8e235e307b2" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d3b7729ac6448e1a27b453a8f278fbc" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 34021 } health_report { overall_health: UNKNOWN } } }
W20260811 12:03:25.920523 22238 tablet.cc:2369] T d25d8c3eff054dbd8555e54403a5fc90 P 3d4a8b75d0574794b67bb8e235e307b2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:03:25.973891 22284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:25.967505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54588 (local address 127.9.221.130:38881)
0811 12:03:25.967801 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:25.967809 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:25.967834 (+ 25us) server_negotiation.cc:407] Connection header received
0811 12:03:25.967921 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:25.967924 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:25.968012 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:25.968175 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:03:25.968756 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.969674 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.971263 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:25.971847 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:25.971999 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:25.972583 (+ 584us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:03:25.973198 (+ 615us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:03:25.973354 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:25.973426 (+ 72us) server_negotiation.cc:299] Negotiation successful
0811 12:03:25.973513 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 12:03:25.983672 21743 raft_consensus.cc:1261] T d25d8c3eff054dbd8555e54403a5fc90 P 5d3b7729ac6448e1a27b453a8f278fbc [term 1 FOLLOWER]: Refusing update from remote peer eae73ca664404c1e9ccc0414bc2daa03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:03:25.983669 22155 raft_consensus.cc:1261] T d25d8c3eff054dbd8555e54403a5fc90 P 3d4a8b75d0574794b67bb8e235e307b2 [term 1 FOLLOWER]: Refusing update from remote peer eae73ca664404c1e9ccc0414bc2daa03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:03:25.984892 22306 consensus_queue.cc:1035] T d25d8c3eff054dbd8555e54403a5fc90 P eae73ca664404c1e9ccc0414bc2daa03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d3b7729ac6448e1a27b453a8f278fbc" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 34021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:03:25.985406 22301 consensus_queue.cc:1035] T d25d8c3eff054dbd8555e54403a5fc90 P eae73ca664404c1e9ccc0414bc2daa03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d4a8b75d0574794b67bb8e235e307b2" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:03:26.023249 20325 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37946:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:03:26.023802 20325 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37946:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:03:26.031574 20325 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5b972c76ffa8496faf82371b3f1523f7: Sending DeleteTablet for 3 replicas of tablet d25d8c3eff054dbd8555e54403a5fc90
I20260811 12:03:26.033248 21717 tablet_service.cc:1514] Processing DeleteTablet for tablet d25d8c3eff054dbd8555e54403a5fc90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:03:26 UTC) from {username='oryx', principal='oryx/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:56782
I20260811 12:03:26.033640 21925 tablet_service.cc:1514] Processing DeleteTablet for tablet d25d8c3eff054dbd8555e54403a5fc90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:03:26 UTC) from {username='oryx', principal='oryx/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:54576
I20260811 12:03:26.033850 22125 tablet_service.cc:1514] Processing DeleteTablet for tablet d25d8c3eff054dbd8555e54403a5fc90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:03:26 UTC) from {username='oryx', principal='oryx/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:43074
I20260811 12:03:26.036207 22334 tablet_replica.cc:330] T d25d8c3eff054dbd8555e54403a5fc90 P eae73ca664404c1e9ccc0414bc2daa03: stopping tablet replica
I20260811 12:03:26.037169 22335 tablet_replica.cc:330] T d25d8c3eff054dbd8555e54403a5fc90 P 3d4a8b75d0574794b67bb8e235e307b2: stopping tablet replica
I20260811 12:03:26.037773 22335 raft_consensus.cc:2229] T d25d8c3eff054dbd8555e54403a5fc90 P 3d4a8b75d0574794b67bb8e235e307b2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:03:26.038689 22335 raft_consensus.cc:2258] T d25d8c3eff054dbd8555e54403a5fc90 P 3d4a8b75d0574794b67bb8e235e307b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:03:26.039095 22331 tablet_replica.cc:330] T d25d8c3eff054dbd8555e54403a5fc90 P 5d3b7729ac6448e1a27b453a8f278fbc: stopping tablet replica
I20260811 12:03:26.039594 22331 raft_consensus.cc:2229] T d25d8c3eff054dbd8555e54403a5fc90 P 5d3b7729ac6448e1a27b453a8f278fbc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:03:26.039626 22334 raft_consensus.cc:2229] T d25d8c3eff054dbd8555e54403a5fc90 P eae73ca664404c1e9ccc0414bc2daa03 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:03:26.040210 22331 raft_consensus.cc:2258] T d25d8c3eff054dbd8555e54403a5fc90 P 5d3b7729ac6448e1a27b453a8f278fbc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:03:26.040786 22334 raft_consensus.cc:2258] T d25d8c3eff054dbd8555e54403a5fc90 P eae73ca664404c1e9ccc0414bc2daa03 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:03:26.042253 22335 ts_tablet_manager.cc:1905] T d25d8c3eff054dbd8555e54403a5fc90 P 3d4a8b75d0574794b67bb8e235e307b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:03:26.042467 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 20281
I20260811 12:03:26.042608 22331 ts_tablet_manager.cc:1905] T d25d8c3eff054dbd8555e54403a5fc90 P 5d3b7729ac6448e1a27b453a8f278fbc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:03:26.043797 20376 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:03:26.050014 22334 ts_tablet_manager.cc:1905] T d25d8c3eff054dbd8555e54403a5fc90 P eae73ca664404c1e9ccc0414bc2daa03: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:03:26.062222 22335 ts_tablet_manager.cc:1918] T d25d8c3eff054dbd8555e54403a5fc90 P 3d4a8b75d0574794b67bb8e235e307b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:03:26.062526 22335 log.cc:1196] T d25d8c3eff054dbd8555e54403a5fc90 P 3d4a8b75d0574794b67bb8e235e307b2: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-2/wal/wals/d25d8c3eff054dbd8555e54403a5fc90
I20260811 12:03:26.063381 22335 ts_tablet_manager.cc:1939] T d25d8c3eff054dbd8555e54403a5fc90 P 3d4a8b75d0574794b67bb8e235e307b2: Deleting consensus metadata
I20260811 12:03:26.068243 22331 ts_tablet_manager.cc:1918] T d25d8c3eff054dbd8555e54403a5fc90 P 5d3b7729ac6448e1a27b453a8f278fbc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:03:26.068507 22331 log.cc:1196] T d25d8c3eff054dbd8555e54403a5fc90 P 5d3b7729ac6448e1a27b453a8f278fbc: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-0/wal/wals/d25d8c3eff054dbd8555e54403a5fc90
I20260811 12:03:26.069370 22331 ts_tablet_manager.cc:1939] T d25d8c3eff054dbd8555e54403a5fc90 P 5d3b7729ac6448e1a27b453a8f278fbc: Deleting consensus metadata
I20260811 12:03:26.074940 22334 ts_tablet_manager.cc:1918] T d25d8c3eff054dbd8555e54403a5fc90 P eae73ca664404c1e9ccc0414bc2daa03: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:03:26.075207 22334 log.cc:1196] T d25d8c3eff054dbd8555e54403a5fc90 P eae73ca664404c1e9ccc0414bc2daa03: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-1/wal/wals/d25d8c3eff054dbd8555e54403a5fc90
I20260811 12:03:26.075690 22243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:26.037054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43714 (local address 127.9.221.189:37829)
0811 12:03:26.037234 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:26.037242 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:26.041627 (+ 4385us) server_negotiation.cc:407] Connection header received
0811 12:03:26.042995 (+ 1368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:26.043003 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:26.043103 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:26.043189 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:26.047460 (+ 4271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:26.049143 (+ 1683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:26.052487 (+ 3344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:26.053534 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:26.053675 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:26.059181 (+ 5506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:26.059189 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:26.059194 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:26.059237 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:26.064201 (+ 4964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:26.072372 (+ 8171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:26.072377 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:26.072381 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:26.072656 (+ 275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:26.073344 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:26.073349 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:26.073352 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:26.073891 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:26.074195 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:26.074970 (+ 775us) server_negotiation.cc:299] Negotiation successful
0811 12:03:26.075247 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 12:03:26.076128 22334 ts_tablet_manager.cc:1939] T d25d8c3eff054dbd8555e54403a5fc90 P eae73ca664404c1e9ccc0414bc2daa03: Deleting consensus metadata
I20260811 12:03:26.097803 22261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:26.041513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55614 (local address 127.9.221.188:34403)
0811 12:03:26.041747 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:26.041755 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:26.077079 (+ 35324us) server_negotiation.cc:407] Connection header received
0811 12:03:26.077275 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:26.077280 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:26.077367 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:26.077480 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:26.080278 (+ 2798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:26.081715 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:26.084129 (+ 2414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:26.085092 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:26.085236 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:26.089417 (+ 4181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:26.089424 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:26.089429 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:26.089470 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:26.094442 (+ 4972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:26.095332 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:26.095338 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:26.095341 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:26.095526 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:26.096001 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:26.096006 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:26.096010 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:26.096378 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:26.096664 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:26.097050 (+ 386us) server_negotiation.cc:299] Negotiation successful
0811 12:03:26.097296 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 12:03:26.201108 20306 catalog_manager.cc:4841] TS 5d3b7729ac6448e1a27b453a8f278fbc (127.9.221.129:34021): tablet d25d8c3eff054dbd8555e54403a5fc90 (table test-table [id=52c7f788b52d49acb67a4cd169d4b49a]) successfully deleted
I20260811 12:03:26.201211 20304 catalog_manager.cc:4841] TS eae73ca664404c1e9ccc0414bc2daa03 (127.9.221.130:38881): tablet d25d8c3eff054dbd8555e54403a5fc90 (table test-table [id=52c7f788b52d49acb67a4cd169d4b49a]) successfully deleted
I20260811 12:03:26.201117 20309 catalog_manager.cc:4841] TS 3d4a8b75d0574794b67bb8e235e307b2 (127.9.221.131:39233): tablet d25d8c3eff054dbd8555e54403a5fc90 (table test-table [id=52c7f788b52d49acb67a4cd169d4b49a]) successfully deleted
I20260811 12:03:26.202030 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 20281
I20260811 12:03:26.262612 22261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:26.243380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55620 (local address 127.9.221.188:34403)
0811 12:03:26.243561 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:26.243568 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:26.244728 (+ 1160us) server_negotiation.cc:407] Connection header received
0811 12:03:26.244880 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:26.244886 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:26.244977 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:26.245104 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:26.248088 (+ 2984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:26.249241 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:26.251264 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:26.252179 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:26.252302 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:26.255364 (+ 3062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:26.255369 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:26.255373 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:26.255432 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:26.259707 (+ 4275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:26.260458 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:26.260464 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:26.260468 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:26.260671 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:26.261100 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:26.261108 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:26.261114 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:26.261426 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:26.261694 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:26.262020 (+ 326us) server_negotiation.cc:299] Negotiation successful
0811 12:03:26.262244 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 12:03:26.263358 22243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:26.242411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43724 (local address 127.9.221.189:37829)
0811 12:03:26.242655 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:26.242663 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:26.243684 (+ 1021us) server_negotiation.cc:407] Connection header received
0811 12:03:26.243971 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:26.243978 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:26.244081 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:26.244190 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:26.246821 (+ 2631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:26.248190 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:26.250010 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:26.250782 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:26.250915 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:26.256062 (+ 5147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:26.256068 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:26.256074 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:26.256117 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:26.259604 (+ 3487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:26.260627 (+ 1023us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:26.260632 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:26.260636 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:26.260786 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:26.261200 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:26.261206 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:26.261219 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:26.261613 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:26.261888 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:26.262694 (+ 806us) server_negotiation.cc:299] Negotiation successful
0811 12:03:26.262928 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 12:03:27.007758 22349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:27.006804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:38537 (local address 127.9.221.129:51341)
0811 12:03:27.007309 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:27.007454 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:38537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":289,"threads_started":1}
W20260811 12:03:27.008263 21808 heartbeater.cc:644] Failed to heartbeat to 127.9.221.190:38537 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.221.190:38537: connect: Connection refused (error 111)
I20260811 12:03:27.012854 22350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:27.012090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:38537 (local address 127.9.221.130:40979)
0811 12:03:27.012560 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:27.012679 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:38537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":261,"threads_started":1}
W20260811 12:03:27.013244 22020 heartbeater.cc:644] Failed to heartbeat to 127.9.221.190:38537 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.221.190:38537: connect: Connection refused (error 111)
I20260811 12:03:27.017839 22352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:27.016904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:38537 (local address 127.9.221.131:37289)
0811 12:03:27.017400 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:27.017591 (+ 191us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:38537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":289,"threads_started":1}
W20260811 12:03:27.018193 22235 heartbeater.cc:644] Failed to heartbeat to 127.9.221.190:38537 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.221.190:38537: connect: Connection refused (error 111)
I20260811 12:03:27.532471 22353 raft_consensus.cc:482] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5b972c76ffa8496faf82371b3f1523f7)
I20260811 12:03:27.532691 22353 raft_consensus.cc:504] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } peers { permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 37829 } attrs { promote: false } } peers { permanent_uuid: "923b96c3241c4a5ab57db67d4d33856b" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 34403 } attrs { promote: false } }
I20260811 12:03:27.533077 22354 raft_consensus.cc:482] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5b972c76ffa8496faf82371b3f1523f7)
I20260811 12:03:27.533324 22354 raft_consensus.cc:504] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } peers { permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 37829 } attrs { promote: false } } peers { permanent_uuid: "923b96c3241c4a5ab57db67d4d33856b" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 34403 } attrs { promote: false } }
I20260811 12:03:27.533596 22353 leader_election.cc:288] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5b972c76ffa8496faf82371b3f1523f7 (127.9.221.190:38537), 37c923a543d6418694238c016281ad2c (127.9.221.189:37829)
I20260811 12:03:27.541010 22357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:27.533795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:38537 (local address 127.0.0.1:38376)
0811 12:03:27.540428 (+ 6633us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:27.540663 (+ 235us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:38537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6471,"threads_started":1}
I20260811 12:03:27.541406 22354 leader_election.cc:288] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5b972c76ffa8496faf82371b3f1523f7 (127.9.221.190:38537), 923b96c3241c4a5ab57db67d4d33856b (127.9.221.188:34403)
W20260811 12:03:27.541671 21511 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.221.190:38537: connect: Connection refused (error 111)
I20260811 12:03:27.551895 22360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:27.535202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:38537 (local address 127.0.0.1:38380)
0811 12:03:27.551115 (+ 15913us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:27.551312 (+ 197us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:38537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13976,"thread_start_us":14109,"threads_started":1}
I20260811 12:03:27.552448 22358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:27.551382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:38537 (local address 127.0.0.1:38398)
0811 12:03:27.551672 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:27.551895 (+ 223us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:38537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
W20260811 12:03:27.552734 21511 leader_election.cc:334] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5b972c76ffa8496faf82371b3f1523f7 (127.9.221.190:38537): Network error: Client connection negotiation failed: client connection to 127.9.221.190:38537: connect: Connection refused (error 111)
W20260811 12:03:27.553560 21223 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.221.190:38537: connect: Connection refused (error 111)
I20260811 12:03:27.555505 22366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:27.554898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:38537 (local address 127.0.0.1:38408)
0811 12:03:27.555106 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:27.555297 (+ 191us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:38537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 12:03:27.555754 21223 leader_election.cc:334] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5b972c76ffa8496faf82371b3f1523f7 (127.9.221.190:38537): Network error: Client connection negotiation failed: client connection to 127.9.221.190:38537: connect: Connection refused (error 111)
I20260811 12:03:27.557483 22357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:27.533997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:37829 (local address 127.0.0.1:43736)
0811 12:03:27.542598 (+ 8601us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:27.542619 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:27.542738 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:27.550251 (+ 7513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:27.550256 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:27.550285 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:03:27.550565 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:27.550580 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:27.552353 (+ 1773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:27.552361 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:27.555187 (+ 2826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:27.555207 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:27.556749 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:27.556755 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:27.556924 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:27.556950 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 12:03:27.557048 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 12:03:27.557177 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8504,"thread_start_us":15428,"threads_started":1}
I20260811 12:03:27.557542 22359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:27.534135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43736 (local address 127.9.221.189:37829)
0811 12:03:27.549783 (+ 15648us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:27.549790 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:27.549807 (+ 17us) server_negotiation.cc:407] Connection header received
0811 12:03:27.549867 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:27.549872 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:27.549937 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:27.550056 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:03:27.550752 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:27.552186 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:27.555380 (+ 3194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:27.556596 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:27.556734 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:27.556832 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:27.557080 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 12:03:27.557207 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15545,"thread_start_us":15162,"threads_started":1}
I20260811 12:03:27.558393 21255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "923b96c3241c4a5ab57db67d4d33856b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "37c923a543d6418694238c016281ad2c" is_pre_election: true
I20260811 12:03:27.558671 21255 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 923b96c3241c4a5ab57db67d4d33856b in term 1.
I20260811 12:03:27.559160 21508 leader_election.cc:302] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 37c923a543d6418694238c016281ad2c, 923b96c3241c4a5ab57db67d4d33856b; no voters: 5b972c76ffa8496faf82371b3f1523f7
I20260811 12:03:27.559499 22353 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 12:03:27.559633 22353 raft_consensus.cc:482] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5b972c76ffa8496faf82371b3f1523f7)
I20260811 12:03:27.559792 22353 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [term 1 FOLLOWER]: Advancing to term 2
I20260811 12:03:27.560055 22360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:27.550981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:34403 (local address 127.0.0.1:55630)
0811 12:03:27.552083 (+ 1102us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:27.552113 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:27.552229 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:27.554363 (+ 2134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:27.554368 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:27.554415 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:03:27.554719 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:27.554730 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:27.556247 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:27.556251 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:27.557890 (+ 1639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:27.557905 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:27.559252 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:27.559259 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:27.559443 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:27.559489 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 12:03:27.559627 (+ 138us) client_negotiation.cc:240] Negotiation successful
0811 12:03:27.559788 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1019,"thread_start_us":2258,"threads_started":1}
I20260811 12:03:27.560192 22365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:27.549893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55630 (local address 127.9.221.188:34403)
0811 12:03:27.553838 (+ 3945us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:27.553842 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:27.553860 (+ 18us) server_negotiation.cc:407] Connection header received
0811 12:03:27.553900 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:27.553903 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:27.553973 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:27.554090 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:03:27.554892 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:27.556089 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:27.558044 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:27.559080 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:27.559174 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:27.559254 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:27.559708 (+ 454us) server_negotiation.cc:299] Negotiation successful
0811 12:03:27.559838 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3814,"thread_start_us":3685,"threads_started":1}
I20260811 12:03:27.560750 21550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "37c923a543d6418694238c016281ad2c" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "923b96c3241c4a5ab57db67d4d33856b" is_pre_election: true
I20260811 12:03:27.562139 22353 raft_consensus.cc:504] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } peers { permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 37829 } attrs { promote: false } } peers { permanent_uuid: "923b96c3241c4a5ab57db67d4d33856b" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 34403 } attrs { promote: false } }
I20260811 12:03:27.562551 21550 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 37c923a543d6418694238c016281ad2c in current term 2: Already voted for candidate 923b96c3241c4a5ab57db67d4d33856b in this term.
I20260811 12:03:27.562772 22353 leader_election.cc:288] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [CANDIDATE]: Term 2 election: Requested vote from peers 5b972c76ffa8496faf82371b3f1523f7 (127.9.221.190:38537), 37c923a543d6418694238c016281ad2c (127.9.221.189:37829)
I20260811 12:03:27.563218 22357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:27.562822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:38537 (local address 127.0.0.1:38422)
0811 12:03:27.563009 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:27.563086 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:38537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260811 12:03:27.563238 21255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "923b96c3241c4a5ab57db67d4d33856b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "37c923a543d6418694238c016281ad2c"
I20260811 12:03:27.563479 21255 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c [term 1 FOLLOWER]: Advancing to term 2
I20260811 12:03:27.563760 21223 leader_election.cc:302] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 37c923a543d6418694238c016281ad2c; no voters: 5b972c76ffa8496faf82371b3f1523f7, 923b96c3241c4a5ab57db67d4d33856b
I20260811 12:03:27.564110 22357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:27.563798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:38537 (local address 127.0.0.1:38432)
0811 12:03:27.563924 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:27.563997 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:38537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 12:03:27.564272 21511 leader_election.cc:334] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 5b972c76ffa8496faf82371b3f1523f7 (127.9.221.190:38537): Network error: Client connection negotiation failed: client connection to 127.9.221.190:38537: connect: Connection refused (error 111)
I20260811 12:03:27.565776 21255 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 923b96c3241c4a5ab57db67d4d33856b in term 2.
I20260811 12:03:27.565939 22354 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 12:03:27.566185 21508 leader_election.cc:302] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 37c923a543d6418694238c016281ad2c, 923b96c3241c4a5ab57db67d4d33856b; no voters: 5b972c76ffa8496faf82371b3f1523f7
I20260811 12:03:27.566474 22353 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [term 2 FOLLOWER]: Leader election won for term 2
I20260811 12:03:27.566897 22353 raft_consensus.cc:686] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [term 2 LEADER]: Becoming Leader. State: Replica: 923b96c3241c4a5ab57db67d4d33856b, State: Running, Role: LEADER
I20260811 12:03:27.567251 22353 consensus_queue.cc:237] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } peers { permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 37829 } attrs { promote: false } } peers { permanent_uuid: "923b96c3241c4a5ab57db67d4d33856b" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 34403 } attrs { promote: false } }
I20260811 12:03:27.569224 22368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 923b96c3241c4a5ab57db67d4d33856b. Latest consensus state: current_term: 2 leader_uuid: "923b96c3241c4a5ab57db67d4d33856b" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } peers { permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 37829 } attrs { promote: false } } peers { permanent_uuid: "923b96c3241c4a5ab57db67d4d33856b" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 34403 } attrs { promote: false } } }
I20260811 12:03:27.569684 22368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [sys.catalog]: This master's current role is: LEADER
I20260811 12:03:27.570276 22372 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:03:27.575102 22372 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:03:27.576222 22372 catalog_manager.cc:1259] Loaded cluster ID: 2bf09098d4e5453ea97c6c9ed333b76a
I20260811 12:03:27.576292 22372 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:03:27.577319 22372 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:03:27.578060 22372 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b: Loaded TSK: 0
I20260811 12:03:27.578984 22372 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:03:27.604194 21529 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55620:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:03:27.611968 21529 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:03:27.615862 22357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:27.615421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:38537 (local address 127.0.0.1:38454)
0811 12:03:27.615586 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:27.615721 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:38537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 12:03:27.615939 21255 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c [term 2 FOLLOWER]: Refusing update from remote peer 923b96c3241c4a5ab57db67d4d33856b: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260811 12:03:27.616815 22368 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [LEADER]: Connected to new peer: Peer: permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 37829 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260811 12:03:27.616966 22357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:27.616608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:38537 (local address 127.0.0.1:38460)
0811 12:03:27.616760 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:27.616832 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:38537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 12:03:27.617170 21511 consensus_peers.cc:487] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b -> Peer 5b972c76ffa8496faf82371b3f1523f7 (127.9.221.190:38537): Couldn't send request to peer 5b972c76ffa8496faf82371b3f1523f7. Status: Network error: Client connection negotiation failed: client connection to 127.9.221.190:38537: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 12:03:27.619741 22354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 923b96c3241c4a5ab57db67d4d33856b. Latest consensus state: current_term: 2 leader_uuid: "923b96c3241c4a5ab57db67d4d33856b" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } peers { permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 37829 } attrs { promote: false } } peers { permanent_uuid: "923b96c3241c4a5ab57db67d4d33856b" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 34403 } attrs { promote: false } } }
I20260811 12:03:27.620055 22354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:03:27.622011 22354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "923b96c3241c4a5ab57db67d4d33856b" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } peers { permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 37829 } attrs { promote: false } } peers { permanent_uuid: "923b96c3241c4a5ab57db67d4d33856b" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 34403 } attrs { promote: false } } }
I20260811 12:03:27.622298 22354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37c923a543d6418694238c016281ad2c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:03:27.621008 22374 mvcc.cc:205] Tried to move back new op lower bound from 7317298411988447232 to 7317298411799158784. Current Snapshot: MvccSnapshot[applied={T|T < 7317298411988447232}]
I20260811 12:03:27.628218 22353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "923b96c3241c4a5ab57db67d4d33856b" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "5b972c76ffa8496faf82371b3f1523f7" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38537 } } peers { permanent_uuid: "37c923a543d6418694238c016281ad2c" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 37829 } attrs { promote: false } } peers { permanent_uuid: "923b96c3241c4a5ab57db67d4d33856b" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 34403 } attrs { promote: false } } }
I20260811 12:03:27.628597 22353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b [sys.catalog]: This master's current role is: LEADER
I20260811 12:03:27.642805 22358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:27.634997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.131:39233 (local address 127.0.0.1:43080)
0811 12:03:27.635569 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:27.635589 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:27.635745 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:27.636090 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:27.636094 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:27.636123 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:03:27.636451 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:27.636461 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:27.637871 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:27.637877 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:27.639788 (+ 1911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:27.639806 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:27.642025 (+ 2219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:27.642038 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:27.642207 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:27.642242 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 12:03:27.642346 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 12:03:27.642492 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"mutex_wait_us":1}
I20260811 12:03:27.642819 22388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:27.635118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43080 (local address 127.9.221.131:39233)
0811 12:03:27.635589 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:27.635595 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:27.635671 (+ 76us) server_negotiation.cc:407] Connection header received
0811 12:03:27.635882 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:27.635887 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:27.635971 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:27.636072 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:03:27.636600 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:27.637737 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:27.639989 (+ 2252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:27.641754 (+ 1765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:27.641916 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:27.642055 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:27.642379 (+ 324us) server_negotiation.cc:299] Negotiation successful
0811 12:03:27.642524 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":257,"threads_started":1}
I20260811 12:03:27.643878 22125 tablet_service.cc:1467] Processing CreateTablet for tablet 25e826bcdeff4fbd8d54ca12afe358c4 (DEFAULT_TABLE table=test-table [id=82d5b124b453425088e66d53356e1ede]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:03:27.644409 22125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25e826bcdeff4fbd8d54ca12afe358c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:03:27.644901 22391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:27.635421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.130:38881 (local address 127.0.0.1:54600)
0811 12:03:27.636816 (+ 1395us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:27.636834 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:27.637237 (+ 403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:27.638139 (+ 902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:27.638145 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:27.638177 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:03:27.638759 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:27.638775 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:27.640170 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:27.640195 (+ 25us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:27.642052 (+ 1857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:27.642064 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:27.644150 (+ 2086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:27.644156 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:27.644324 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:27.644346 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 12:03:27.644451 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 12:03:27.644610 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1217}
I20260811 12:03:27.644920 22389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:27.635761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54600 (local address 127.9.221.130:38881)
0811 12:03:27.636223 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:27.636231 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:27.637095 (+ 864us) server_negotiation.cc:407] Connection header received
0811 12:03:27.637425 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:27.637434 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:27.637549 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:27.637840 (+ 291us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:03:27.638983 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:27.640052 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:27.642207 (+ 2155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:27.643941 (+ 1734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:27.644085 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:27.644194 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:27.644486 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 12:03:27.644632 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":1024,"threads_started":1}
I20260811 12:03:27.645670 21925 tablet_service.cc:1467] Processing CreateTablet for tablet 25e826bcdeff4fbd8d54ca12afe358c4 (DEFAULT_TABLE table=test-table [id=82d5b124b453425088e66d53356e1ede]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:03:27.646107 21925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25e826bcdeff4fbd8d54ca12afe358c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:03:27.651629 22357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:27.636168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:34021 (local address 127.0.0.1:56792)
0811 12:03:27.638111 (+ 1943us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:27.638132 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:27.638246 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:27.645560 (+ 7314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:27.645564 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:27.645590 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:03:27.645743 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:27.645753 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:27.647136 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:27.647141 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:27.649001 (+ 1860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:27.649018 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:27.650865 (+ 1847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:27.650871 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:27.651066 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:27.651100 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 12:03:27.651205 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 12:03:27.651332 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1815,"thread_start_us":726,"threads_started":1}
I20260811 12:03:27.651916 22392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:27.636632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56792 (local address 127.9.221.129:34021)
0811 12:03:27.644100 (+ 7468us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:27.644105 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:27.644125 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:03:27.644194 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:27.644201 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:27.644275 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:27.645531 (+ 1256us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:03:27.645901 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:27.646973 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:27.649176 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:27.650676 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:27.650847 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:27.650997 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:27.651261 (+ 264us) server_negotiation.cc:299] Negotiation successful
0811 12:03:27.651465 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7346,"thread_start_us":7252,"threads_started":1}
I20260811 12:03:27.652869 21717 tablet_service.cc:1467] Processing CreateTablet for tablet 25e826bcdeff4fbd8d54ca12afe358c4 (DEFAULT_TABLE table=test-table [id=82d5b124b453425088e66d53356e1ede]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:03:27.653415 21717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25e826bcdeff4fbd8d54ca12afe358c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:03:27.656392 22393 tablet_bootstrap.cc:492] T 25e826bcdeff4fbd8d54ca12afe358c4 P 3d4a8b75d0574794b67bb8e235e307b2: Bootstrap starting.
I20260811 12:03:27.656559 22396 tablet_bootstrap.cc:492] T 25e826bcdeff4fbd8d54ca12afe358c4 P eae73ca664404c1e9ccc0414bc2daa03: Bootstrap starting.
I20260811 12:03:27.658077 22396 tablet_bootstrap.cc:654] T 25e826bcdeff4fbd8d54ca12afe358c4 P eae73ca664404c1e9ccc0414bc2daa03: Neither blocks nor log segments found. Creating new log.
I20260811 12:03:27.658176 22393 tablet_bootstrap.cc:654] T 25e826bcdeff4fbd8d54ca12afe358c4 P 3d4a8b75d0574794b67bb8e235e307b2: Neither blocks nor log segments found. Creating new log.
I20260811 12:03:27.663580 22393 tablet_bootstrap.cc:492] T 25e826bcdeff4fbd8d54ca12afe358c4 P 3d4a8b75d0574794b67bb8e235e307b2: No bootstrap required, opened a new log
I20260811 12:03:27.663902 22393 ts_tablet_manager.cc:1397] T 25e826bcdeff4fbd8d54ca12afe358c4 P 3d4a8b75d0574794b67bb8e235e307b2: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.001s
I20260811 12:03:27.665048 22393 raft_consensus.cc:348] T 25e826bcdeff4fbd8d54ca12afe358c4 P 3d4a8b75d0574794b67bb8e235e307b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d4a8b75d0574794b67bb8e235e307b2" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39233 } } peers { permanent_uuid: "eae73ca664404c1e9ccc0414bc2daa03" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38881 } } peers { permanent_uuid: "5d3b7729ac6448e1a27b453a8f278fbc" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 34021 } }
I20260811 12:03:27.665380 22393 raft_consensus.cc:374] T 25e826bcdeff4fbd8d54ca12afe358c4 P 3d4a8b75d0574794b67bb8e235e307b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:03:27.665514 22393 raft_consensus.cc:729] T 25e826bcdeff4fbd8d54ca12afe358c4 P 3d4a8b75d0574794b67bb8e235e307b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d4a8b75d0574794b67bb8e235e307b2, State: Initialized, Role: FOLLOWER
I20260811 12:03:27.665825 22393 consensus_queue.cc:260] T 25e826bcdeff4fbd8d54ca12afe358c4 P 3d4a8b75d0574794b67bb8e235e307b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d4a8b75d0574794b67bb8e235e307b2" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39233 } } peers { permanent_uuid: "eae73ca664404c1e9ccc0414bc2daa03" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38881 } } peers { permanent_uuid: "5d3b7729ac6448e1a27b453a8f278fbc" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 34021 } }
I20260811 12:03:27.666878 22393 ts_tablet_manager.cc:1428] T 25e826bcdeff4fbd8d54ca12afe358c4 P 3d4a8b75d0574794b67bb8e235e307b2: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 12:03:27.673771 22404 tablet_bootstrap.cc:492] T 25e826bcdeff4fbd8d54ca12afe358c4 P 5d3b7729ac6448e1a27b453a8f278fbc: Bootstrap starting.
I20260811 12:03:27.676376 22404 tablet_bootstrap.cc:654] T 25e826bcdeff4fbd8d54ca12afe358c4 P 5d3b7729ac6448e1a27b453a8f278fbc: Neither blocks nor log segments found. Creating new log.
I20260811 12:03:27.680764 22234 heartbeater.cc:503] Master 127.9.221.188:34403 was elected leader, sending a full tablet report...
I20260811 12:03:27.681063 22401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:27.667522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:38537 (local address 127.9.221.131:34543)
0811 12:03:27.680600 (+ 13078us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:27.680780 (+ 180us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:38537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12924,"thread_start_us":12800,"threads_started":1}
I20260811 12:03:27.682813 22396 tablet_bootstrap.cc:492] T 25e826bcdeff4fbd8d54ca12afe358c4 P eae73ca664404c1e9ccc0414bc2daa03: No bootstrap required, opened a new log
I20260811 12:03:27.682947 22401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:27.682261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:38537 (local address 127.9.221.131:34495)
0811 12:03:27.682723 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:27.682811 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:38537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347}
I20260811 12:03:27.682977 22396 ts_tablet_manager.cc:1397] T 25e826bcdeff4fbd8d54ca12afe358c4 P eae73ca664404c1e9ccc0414bc2daa03: Time spent bootstrapping tablet: real 0.027s user 0.012s sys 0.013s
I20260811 12:03:27.683842 22404 tablet_bootstrap.cc:492] T 25e826bcdeff4fbd8d54ca12afe358c4 P 5d3b7729ac6448e1a27b453a8f278fbc: No bootstrap required, opened a new log
I20260811 12:03:27.684012 22404 ts_tablet_manager.cc:1397] T 25e826bcdeff4fbd8d54ca12afe358c4 P 5d3b7729ac6448e1a27b453a8f278fbc: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260811 12:03:27.683939 22396 raft_consensus.cc:348] T 25e826bcdeff4fbd8d54ca12afe358c4 P eae73ca664404c1e9ccc0414bc2daa03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d4a8b75d0574794b67bb8e235e307b2" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39233 } } peers { permanent_uuid: "eae73ca664404c1e9ccc0414bc2daa03" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38881 } } peers { permanent_uuid: "5d3b7729ac6448e1a27b453a8f278fbc" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 34021 } }
I20260811 12:03:27.684191 22396 raft_consensus.cc:374] T 25e826bcdeff4fbd8d54ca12afe358c4 P eae73ca664404c1e9ccc0414bc2daa03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:03:27.684299 22396 raft_consensus.cc:729] T 25e826bcdeff4fbd8d54ca12afe358c4 P eae73ca664404c1e9ccc0414bc2daa03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eae73ca664404c1e9ccc0414bc2daa03, State: Initialized, Role: FOLLOWER
I20260811 12:03:27.684540 22396 consensus_queue.cc:260] T 25e826bcdeff4fbd8d54ca12afe358c4 P eae73ca664404c1e9ccc0414bc2daa03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d4a8b75d0574794b67bb8e235e307b2" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39233 } } peers { permanent_uuid: "eae73ca664404c1e9ccc0414bc2daa03" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38881 } } peers { permanent_uuid: "5d3b7729ac6448e1a27b453a8f278fbc" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 34021 } }
I20260811 12:03:27.684839 22404 raft_consensus.cc:348] T 25e826bcdeff4fbd8d54ca12afe358c4 P 5d3b7729ac6448e1a27b453a8f278fbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d4a8b75d0574794b67bb8e235e307b2" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39233 } } peers { permanent_uuid: "eae73ca664404c1e9ccc0414bc2daa03" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38881 } } peers { permanent_uuid: "5d3b7729ac6448e1a27b453a8f278fbc" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 34021 } }
I20260811 12:03:27.685117 22404 raft_consensus.cc:374] T 25e826bcdeff4fbd8d54ca12afe358c4 P 5d3b7729ac6448e1a27b453a8f278fbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:03:27.685216 22404 raft_consensus.cc:729] T 25e826bcdeff4fbd8d54ca12afe358c4 P 5d3b7729ac6448e1a27b453a8f278fbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d3b7729ac6448e1a27b453a8f278fbc, State: Initialized, Role: FOLLOWER
I20260811 12:03:27.685397 22396 ts_tablet_manager.cc:1428] T 25e826bcdeff4fbd8d54ca12afe358c4 P eae73ca664404c1e9ccc0414bc2daa03: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 12:03:27.685477 22404 consensus_queue.cc:260] T 25e826bcdeff4fbd8d54ca12afe358c4 P 5d3b7729ac6448e1a27b453a8f278fbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d4a8b75d0574794b67bb8e235e307b2" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39233 } } peers { permanent_uuid: "eae73ca664404c1e9ccc0414bc2daa03" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38881 } } peers { permanent_uuid: "5d3b7729ac6448e1a27b453a8f278fbc" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 34021 } }
I20260811 12:03:27.686746 22404 ts_tablet_manager.cc:1428] T 25e826bcdeff4fbd8d54ca12afe358c4 P 5d3b7729ac6448e1a27b453a8f278fbc: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 12:03:27.688246 22018 heartbeater.cc:503] Master 127.9.221.188:34403 was elected leader, sending a full tablet report...
I20260811 12:03:27.688271 22413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:27.687094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:38537 (local address 127.9.221.130:55691)
0811 12:03:27.687783 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:27.687960 (+ 177us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:38537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":359,"threads_started":1}
I20260811 12:03:27.690306 22411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:27.686688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:38537 (local address 127.9.221.129:36479)
0811 12:03:27.689852 (+ 3164us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:27.690039 (+ 187us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:38537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3026,"thread_start_us":2375,"threads_started":1}
I20260811 12:03:27.691290 22411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:27.690945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:38537 (local address 127.9.221.129:54145)
0811 12:03:27.691090 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:27.691172 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:38537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 12:03:27.691326 21807 heartbeater.cc:503] Master 127.9.221.188:34403 was elected leader, sending a full tablet report...
I20260811 12:03:27.731446 22400 raft_consensus.cc:482] T 25e826bcdeff4fbd8d54ca12afe358c4 P 3d4a8b75d0574794b67bb8e235e307b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:03:27.731776 22400 raft_consensus.cc:504] T 25e826bcdeff4fbd8d54ca12afe358c4 P 3d4a8b75d0574794b67bb8e235e307b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d4a8b75d0574794b67bb8e235e307b2" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39233 } } peers { permanent_uuid: "eae73ca664404c1e9ccc0414bc2daa03" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38881 } } peers { permanent_uuid: "5d3b7729ac6448e1a27b453a8f278fbc" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 34021 } }
I20260811 12:03:27.733592 22400 leader_election.cc:288] T 25e826bcdeff4fbd8d54ca12afe358c4 P 3d4a8b75d0574794b67bb8e235e307b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eae73ca664404c1e9ccc0414bc2daa03 (127.9.221.130:38881), 5d3b7729ac6448e1a27b453a8f278fbc (127.9.221.129:34021)
I20260811 12:03:27.740716 22389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:27.734289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:38859 (local address 127.9.221.130:38881)
0811 12:03:27.734485 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:27.734489 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:27.734504 (+ 15us) server_negotiation.cc:407] Connection header received
0811 12:03:27.734562 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:27.734566 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:27.734622 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:27.734736 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:03:27.735263 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:27.736379 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:27.738486 (+ 2107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:27.739786 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:27.739983 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:27.740062 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:27.740423 (+ 361us) server_negotiation.cc:299] Negotiation successful
0811 12:03:27.740527 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 12:03:27.740828 22401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:27.733843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.130:38881 (local address 127.9.221.131:38859)
0811 12:03:27.734082 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:27.734107 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:27.734281 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:27.734758 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:27.734762 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:27.734795 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:03:27.735066 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:27.735076 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:27.736547 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:27.736553 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:27.738323 (+ 1770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:27.738335 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:27.740055 (+ 1720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:27.740062 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:27.740238 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:27.740273 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 12:03:27.740379 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 12:03:27.740519 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260811 12:03:27.741822 22392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:27.734090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:56087 (local address 127.9.221.129:34021)
0811 12:03:27.734285 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:27.734289 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:27.735180 (+ 891us) server_negotiation.cc:407] Connection header received
0811 12:03:27.735371 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:27.735374 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:27.735432 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:27.735501 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:03:27.736197 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:27.737303 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:27.739565 (+ 2262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:27.740722 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:27.740925 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:27.741000 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:27.741430 (+ 430us) server_negotiation.cc:299] Negotiation successful
0811 12:03:27.741560 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 12:03:27.741719 21951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25e826bcdeff4fbd8d54ca12afe358c4" candidate_uuid: "3d4a8b75d0574794b67bb8e235e307b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eae73ca664404c1e9ccc0414bc2daa03" is_pre_election: true
I20260811 12:03:27.741914 22414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:27.733843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:34021 (local address 127.9.221.131:56087)
0811 12:03:27.735058 (+ 1215us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:27.735074 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:27.735225 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:27.735560 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:27.735565 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:27.735596 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:03:27.735997 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:27.736010 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:27.737440 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:27.737444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:27.739393 (+ 1949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:27.739408 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:27.741095 (+ 1687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:27.741101 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:27.741265 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:27.741288 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 12:03:27.741403 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 12:03:27.741558 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1077,"mutex_wait_us":188,"thread_start_us":692,"threads_started":1}
I20260811 12:03:27.742123 21951 raft_consensus.cc:2454] T 25e826bcdeff4fbd8d54ca12afe358c4 P eae73ca664404c1e9ccc0414bc2daa03 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d4a8b75d0574794b67bb8e235e307b2 in term 0.
I20260811 12:03:27.742743 22057 leader_election.cc:302] T 25e826bcdeff4fbd8d54ca12afe358c4 P 3d4a8b75d0574794b67bb8e235e307b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d4a8b75d0574794b67bb8e235e307b2, eae73ca664404c1e9ccc0414bc2daa03; no voters:
I20260811 12:03:27.742712 21743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25e826bcdeff4fbd8d54ca12afe358c4" candidate_uuid: "3d4a8b75d0574794b67bb8e235e307b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d3b7729ac6448e1a27b453a8f278fbc" is_pre_election: true
I20260811 12:03:27.742995 21743 raft_consensus.cc:2454] T 25e826bcdeff4fbd8d54ca12afe358c4 P 5d3b7729ac6448e1a27b453a8f278fbc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d4a8b75d0574794b67bb8e235e307b2 in term 0.
I20260811 12:03:27.743175 22400 raft_consensus.cc:2781] T 25e826bcdeff4fbd8d54ca12afe358c4 P 3d4a8b75d0574794b67bb8e235e307b2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:03:27.743325 22400 raft_consensus.cc:482] T 25e826bcdeff4fbd8d54ca12afe358c4 P 3d4a8b75d0574794b67bb8e235e307b2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:03:27.743441 22400 raft_consensus.cc:3037] T 25e826bcdeff4fbd8d54ca12afe358c4 P 3d4a8b75d0574794b67bb8e235e307b2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:03:27.745707 22400 raft_consensus.cc:504] T 25e826bcdeff4fbd8d54ca12afe358c4 P 3d4a8b75d0574794b67bb8e235e307b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d4a8b75d0574794b67bb8e235e307b2" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39233 } } peers { permanent_uuid: "eae73ca664404c1e9ccc0414bc2daa03" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38881 } } peers { permanent_uuid: "5d3b7729ac6448e1a27b453a8f278fbc" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 34021 } }
I20260811 12:03:27.746320 22400 leader_election.cc:288] T 25e826bcdeff4fbd8d54ca12afe358c4 P 3d4a8b75d0574794b67bb8e235e307b2 [CANDIDATE]: Term 1 election: Requested vote from peers eae73ca664404c1e9ccc0414bc2daa03 (127.9.221.130:38881), 5d3b7729ac6448e1a27b453a8f278fbc (127.9.221.129:34021)
I20260811 12:03:27.746860 21951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25e826bcdeff4fbd8d54ca12afe358c4" candidate_uuid: "3d4a8b75d0574794b67bb8e235e307b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eae73ca664404c1e9ccc0414bc2daa03"
I20260811 12:03:27.746860 21743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25e826bcdeff4fbd8d54ca12afe358c4" candidate_uuid: "3d4a8b75d0574794b67bb8e235e307b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d3b7729ac6448e1a27b453a8f278fbc"
I20260811 12:03:27.747081 21951 raft_consensus.cc:3037] T 25e826bcdeff4fbd8d54ca12afe358c4 P eae73ca664404c1e9ccc0414bc2daa03 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:03:27.747084 21743 raft_consensus.cc:3037] T 25e826bcdeff4fbd8d54ca12afe358c4 P 5d3b7729ac6448e1a27b453a8f278fbc [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:03:27.750326 21743 raft_consensus.cc:2454] T 25e826bcdeff4fbd8d54ca12afe358c4 P 5d3b7729ac6448e1a27b453a8f278fbc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d4a8b75d0574794b67bb8e235e307b2 in term 1.
I20260811 12:03:27.750348 21951 raft_consensus.cc:2454] T 25e826bcdeff4fbd8d54ca12afe358c4 P eae73ca664404c1e9ccc0414bc2daa03 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d4a8b75d0574794b67bb8e235e307b2 in term 1.
I20260811 12:03:27.750765 22058 leader_election.cc:302] T 25e826bcdeff4fbd8d54ca12afe358c4 P 3d4a8b75d0574794b67bb8e235e307b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3d4a8b75d0574794b67bb8e235e307b2, 5d3b7729ac6448e1a27b453a8f278fbc, eae73ca664404c1e9ccc0414bc2daa03; no voters:
I20260811 12:03:27.751183 22400 raft_consensus.cc:2781] T 25e826bcdeff4fbd8d54ca12afe358c4 P 3d4a8b75d0574794b67bb8e235e307b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:03:27.751708 22400 raft_consensus.cc:686] T 25e826bcdeff4fbd8d54ca12afe358c4 P 3d4a8b75d0574794b67bb8e235e307b2 [term 1 LEADER]: Becoming Leader. State: Replica: 3d4a8b75d0574794b67bb8e235e307b2, State: Running, Role: LEADER
I20260811 12:03:27.752177 22400 consensus_queue.cc:237] T 25e826bcdeff4fbd8d54ca12afe358c4 P 3d4a8b75d0574794b67bb8e235e307b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d4a8b75d0574794b67bb8e235e307b2" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39233 } } peers { permanent_uuid: "eae73ca664404c1e9ccc0414bc2daa03" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38881 } } peers { permanent_uuid: "5d3b7729ac6448e1a27b453a8f278fbc" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 34021 } }
I20260811 12:03:27.752779 22414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:27.752374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:38537 (local address 127.9.221.131:58309)
0811 12:03:27.752560 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:27.752654 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:38537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 12:03:27.756357 21529 catalog_manager.cc:5495] T 25e826bcdeff4fbd8d54ca12afe358c4 P 3d4a8b75d0574794b67bb8e235e307b2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d4a8b75d0574794b67bb8e235e307b2 (127.9.221.131). New cstate: current_term: 1 leader_uuid: "3d4a8b75d0574794b67bb8e235e307b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d4a8b75d0574794b67bb8e235e307b2" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 39233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eae73ca664404c1e9ccc0414bc2daa03" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d3b7729ac6448e1a27b453a8f278fbc" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 34021 } health_report { overall_health: UNKNOWN } } }
I20260811 12:03:27.797394 22388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:27.792381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43096 (local address 127.9.221.131:39233)
0811 12:03:27.792623 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:27.792628 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:27.792644 (+ 16us) server_negotiation.cc:407] Connection header received
0811 12:03:27.792730 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:27.792751 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:27.792829 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:27.792993 (+ 164us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:03:27.793309 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:27.794003 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:27.795202 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:27.795616 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:27.795782 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:27.796192 (+ 410us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:03:27.796759 (+ 567us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:03:27.796932 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:27.797038 (+ 106us) server_negotiation.cc:299] Negotiation successful
0811 12:03:27.797124 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
W20260811 12:03:27.800748 22022 tablet.cc:2369] T 25e826bcdeff4fbd8d54ca12afe358c4 P eae73ca664404c1e9ccc0414bc2daa03: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:03:27.801821 21743 raft_consensus.cc:1261] T 25e826bcdeff4fbd8d54ca12afe358c4 P 5d3b7729ac6448e1a27b453a8f278fbc [term 1 FOLLOWER]: Refusing update from remote peer 3d4a8b75d0574794b67bb8e235e307b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:03:27.802067 21951 raft_consensus.cc:1261] T 25e826bcdeff4fbd8d54ca12afe358c4 P eae73ca664404c1e9ccc0414bc2daa03 [term 1 FOLLOWER]: Refusing update from remote peer 3d4a8b75d0574794b67bb8e235e307b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:03:27.802644 22416 consensus_queue.cc:1035] T 25e826bcdeff4fbd8d54ca12afe358c4 P 3d4a8b75d0574794b67bb8e235e307b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d3b7729ac6448e1a27b453a8f278fbc" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 34021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:03:27.803093 22400 consensus_queue.cc:1035] T 25e826bcdeff4fbd8d54ca12afe358c4 P 3d4a8b75d0574794b67bb8e235e307b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eae73ca664404c1e9ccc0414bc2daa03" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:03:27.805876 22411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:27.805147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:38537 (local address 127.9.221.129:47825)
0811 12:03:27.805445 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:27.805590 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:38537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151}
I20260811 12:03:27.806202 22413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:27.805814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:38537 (local address 127.9.221.130:33573)
0811 12:03:27.805981 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:27.806070 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:38537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 12:03:27.812170 22411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:27.811550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:38537 (local address 127.9.221.129:36065)
0811 12:03:27.811817 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:27.811933 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:38537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260811 12:03:27.815217 22414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:27.814631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:38537 (local address 127.9.221.131:45499)
0811 12:03:27.814937 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:27.815040 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:38537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202}
I20260811 12:03:27.817708 22421 mvcc.cc:205] Tried to move back new op lower bound from 7317298412748210176 to 7317298412557434880. Current Snapshot: MvccSnapshot[applied={T|T < 7317298412748210176 or (T in {7317298412748210176})}]
I20260811 12:03:27.824175 22424 mvcc.cc:205] Tried to move back new op lower bound from 7317298412748210176 to 7317298412557434880. Current Snapshot: MvccSnapshot[applied={T|T < 7317298412748210176}]
I20260811 12:03:27.825347 22413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:27.824677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:38537 (local address 127.9.221.130:36337)
0811 12:03:27.824958 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:27.825094 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:38537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
I20260811 12:03:27.827587 22414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:27.826937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:38537 (local address 127.9.221.131:44327)
0811 12:03:27.827224 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:27.827371 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:38537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
I20260811 12:03:27.841204 21532 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55620:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:03:27.841572 21532 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55620:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:03:27.847326 21532 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 923b96c3241c4a5ab57db67d4d33856b: Sending DeleteTablet for 3 replicas of tablet 25e826bcdeff4fbd8d54ca12afe358c4
I20260811 12:03:27.848366 22125 tablet_service.cc:1514] Processing DeleteTablet for tablet 25e826bcdeff4fbd8d54ca12afe358c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:03:27 UTC) from {username='oryx', principal='oryx/127.9.221.188@KRBTEST.COM'} at 127.0.0.1:43080
I20260811 12:03:27.848685 21717 tablet_service.cc:1514] Processing DeleteTablet for tablet 25e826bcdeff4fbd8d54ca12afe358c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:03:27 UTC) from {username='oryx', principal='oryx/127.9.221.188@KRBTEST.COM'} at 127.0.0.1:56792
I20260811 12:03:27.848786 21925 tablet_service.cc:1514] Processing DeleteTablet for tablet 25e826bcdeff4fbd8d54ca12afe358c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:03:27 UTC) from {username='oryx', principal='oryx/127.9.221.188@KRBTEST.COM'} at 127.0.0.1:54600
I20260811 12:03:27.850904 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 21491
I20260811 12:03:27.860472 22443 tablet_replica.cc:330] T 25e826bcdeff4fbd8d54ca12afe358c4 P 5d3b7729ac6448e1a27b453a8f278fbc: stopping tablet replica
I20260811 12:03:27.860728 22443 raft_consensus.cc:2229] T 25e826bcdeff4fbd8d54ca12afe358c4 P 5d3b7729ac6448e1a27b453a8f278fbc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:03:27.861092 22443 raft_consensus.cc:2258] T 25e826bcdeff4fbd8d54ca12afe358c4 P 5d3b7729ac6448e1a27b453a8f278fbc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:03:27.861804 22441 tablet_replica.cc:330] T 25e826bcdeff4fbd8d54ca12afe358c4 P 3d4a8b75d0574794b67bb8e235e307b2: stopping tablet replica
I20260811 12:03:27.862030 22441 raft_consensus.cc:2229] T 25e826bcdeff4fbd8d54ca12afe358c4 P 3d4a8b75d0574794b67bb8e235e307b2 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:03:27.862782 22441 raft_consensus.cc:2258] T 25e826bcdeff4fbd8d54ca12afe358c4 P 3d4a8b75d0574794b67bb8e235e307b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:03:27.864004 22441 ts_tablet_manager.cc:1905] T 25e826bcdeff4fbd8d54ca12afe358c4 P 3d4a8b75d0574794b67bb8e235e307b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:03:27.870110 22443 ts_tablet_manager.cc:1905] T 25e826bcdeff4fbd8d54ca12afe358c4 P 5d3b7729ac6448e1a27b453a8f278fbc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:03:27.871500 22444 tablet_replica.cc:330] T 25e826bcdeff4fbd8d54ca12afe358c4 P eae73ca664404c1e9ccc0414bc2daa03: stopping tablet replica
I20260811 12:03:27.871726 22444 raft_consensus.cc:2229] T 25e826bcdeff4fbd8d54ca12afe358c4 P eae73ca664404c1e9ccc0414bc2daa03 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:03:27.872088 22444 raft_consensus.cc:2258] T 25e826bcdeff4fbd8d54ca12afe358c4 P eae73ca664404c1e9ccc0414bc2daa03 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:03:27.875422 22444 ts_tablet_manager.cc:1905] T 25e826bcdeff4fbd8d54ca12afe358c4 P eae73ca664404c1e9ccc0414bc2daa03: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:03:27.875490 22441 ts_tablet_manager.cc:1918] T 25e826bcdeff4fbd8d54ca12afe358c4 P 3d4a8b75d0574794b67bb8e235e307b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:03:27.875823 22441 log.cc:1196] T 25e826bcdeff4fbd8d54ca12afe358c4 P 3d4a8b75d0574794b67bb8e235e307b2: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-2/wal/wals/25e826bcdeff4fbd8d54ca12afe358c4
I20260811 12:03:27.876343 22441 ts_tablet_manager.cc:1939] T 25e826bcdeff4fbd8d54ca12afe358c4 P 3d4a8b75d0574794b67bb8e235e307b2: Deleting consensus metadata
I20260811 12:03:27.877516 21511 catalog_manager.cc:4841] TS 3d4a8b75d0574794b67bb8e235e307b2 (127.9.221.131:39233): tablet 25e826bcdeff4fbd8d54ca12afe358c4 (table test-table [id=82d5b124b453425088e66d53356e1ede]) successfully deleted
I20260811 12:03:27.880923 22444 ts_tablet_manager.cc:1918] T 25e826bcdeff4fbd8d54ca12afe358c4 P eae73ca664404c1e9ccc0414bc2daa03: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:03:27.881067 22444 log.cc:1196] T 25e826bcdeff4fbd8d54ca12afe358c4 P eae73ca664404c1e9ccc0414bc2daa03: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-1/wal/wals/25e826bcdeff4fbd8d54ca12afe358c4
I20260811 12:03:27.881542 22444 ts_tablet_manager.cc:1939] T 25e826bcdeff4fbd8d54ca12afe358c4 P eae73ca664404c1e9ccc0414bc2daa03: Deleting consensus metadata
I20260811 12:03:27.882603 21508 catalog_manager.cc:4841] TS eae73ca664404c1e9ccc0414bc2daa03 (127.9.221.130:38881): tablet 25e826bcdeff4fbd8d54ca12afe358c4 (table test-table [id=82d5b124b453425088e66d53356e1ede]) successfully deleted
I20260811 12:03:27.884327 22443 ts_tablet_manager.cc:1918] T 25e826bcdeff4fbd8d54ca12afe358c4 P 5d3b7729ac6448e1a27b453a8f278fbc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:03:27.884485 22443 log.cc:1196] T 25e826bcdeff4fbd8d54ca12afe358c4 P 5d3b7729ac6448e1a27b453a8f278fbc: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449743084027-10102-0/minicluster-data/ts-0/wal/wals/25e826bcdeff4fbd8d54ca12afe358c4
I20260811 12:03:27.884976 22443 ts_tablet_manager.cc:1939] T 25e826bcdeff4fbd8d54ca12afe358c4 P 5d3b7729ac6448e1a27b453a8f278fbc: Deleting consensus metadata
I20260811 12:03:27.885977 21509 catalog_manager.cc:4841] TS 5d3b7729ac6448e1a27b453a8f278fbc (127.9.221.129:34021): tablet 25e826bcdeff4fbd8d54ca12afe358c4 (table test-table [id=82d5b124b453425088e66d53356e1ede]) successfully deleted
I20260811 12:03:27.891341 22392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:27.852389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56796 (local address 127.9.221.129:34021)
0811 12:03:27.852626 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:27.852634 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:27.861974 (+ 9340us) server_negotiation.cc:407] Connection header received
0811 12:03:27.862149 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:27.862157 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:27.862239 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:27.862368 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:27.864374 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:27.865480 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:27.867348 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:27.868142 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:27.868283 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:27.883812 (+ 15529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:27.883830 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:27.883848 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:27.883943 (+ 95us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:27.888166 (+ 4223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:27.889019 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:27.889031 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:27.889035 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:27.889196 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:27.889615 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:27.889620 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:27.889624 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:27.890108 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:27.890398 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:27.890712 (+ 314us) server_negotiation.cc:299] Negotiation successful
0811 12:03:27.890976 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 12:03:27.892005 21801 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:03:28.006728 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 21491
I20260811 12:03:28.034291 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 21826
I20260811 12:03:28.041824 22357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:28.041316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:38537 (local address 127.0.0.1:38462)
0811 12:03:28.041502 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:28.041610 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:38537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 12:03:28.043035 22357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:28.042597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:38537 (local address 127.0.0.1:38470)
0811 12:03:28.042775 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:28.042873 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:38537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 12:03:28.050136 22389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:28.035144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54606 (local address 127.9.221.130:38881)
0811 12:03:28.035342 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:28.035347 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:28.035392 (+ 45us) server_negotiation.cc:407] Connection header received
0811 12:03:28.035587 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:28.035591 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:28.035709 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:28.035828 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:28.037621 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:28.038366 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:28.040150 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:28.040695 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:28.040849 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:28.043973 (+ 3124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:28.043993 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:28.044008 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:28.044039 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:28.047105 (+ 3066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:28.047896 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:28.047904 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:28.047906 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:28.048026 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:28.048418 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:28.048422 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:28.048424 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:28.048877 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:28.049205 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:28.049523 (+ 318us) server_negotiation.cc:299] Negotiation successful
0811 12:03:28.049754 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 12:03:28.050834 22012 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:03:28.174279 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 21826
I20260811 12:03:28.208335 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 22035
I20260811 12:03:28.223395 22388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:28.209202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43106 (local address 127.9.221.131:39233)
0811 12:03:28.209375 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:28.209381 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:28.209430 (+ 49us) server_negotiation.cc:407] Connection header received
0811 12:03:28.209662 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:28.209672 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:28.209766 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:28.209875 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:28.211526 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:28.212496 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:28.213714 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:28.214162 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:28.214326 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:28.216988 (+ 2662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:28.217015 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:28.217034 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:28.217063 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:28.220463 (+ 3400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:28.221210 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:28.221219 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:28.221223 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:28.221399 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:28.221801 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:28.221806 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:28.221813 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:28.222254 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:28.222600 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:28.222833 (+ 233us) server_negotiation.cc:299] Negotiation successful
0811 12:03:28.223057 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 12:03:28.224159 22225 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:03:28.357266 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 22035
I20260811 12:03:28.386502 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 21021
I20260811 12:03:28.387616 21292 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:03:28.508915 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 21021
I20260811 12:03:28.537425 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 21177
I20260811 12:03:28.538653 21582 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:03:28.539109 22357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:28.538538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:38537 (local address 127.0.0.1:38480)
0811 12:03:28.538747 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:28.538852 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:38537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 12:03:28.665923 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 21177
I20260811 12:03:28.665938 22357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:28.665378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:38537 (local address 127.0.0.1:38496)
0811 12:03:28.665589 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:28.665734 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:38537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
2026-08-11T12:03:28Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (10357 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:03:28 dist-test-slave-mjjr krb5kdc[22459](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:03:28 dist-test-slave-mjjr krb5kdc[22459](info): set up 2 sockets
Aug 11 12:03:28 dist-test-slave-mjjr krb5kdc[22459](info): commencing operation
krb5kdc: starting...
W20260811 12:03:30.022933 10102 mini_kdc.cc:121] Time spent starting KDC: real 1.301s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Aug 11 12:03:30 dist-test-slave-mjjr krb5kdc[22459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449810, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:03:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:03:30Z Disabled control of system clock
WARNING: no policy specified for oryx/127.9.221.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.221.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.221.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.190.keytab.
Entry for principal oryx/127.9.221.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.190.keytab.
I20260811 12:03:30.248629 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:35905
--webserver_interface=127.9.221.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.190.keytab
--principal=oryx/127.9.221.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:39085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:35905
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:03:30.575798 22481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:03:30.576086 22481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:03:30.576241 22481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:03:30.583604 22481 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:03:30.583750 22481 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 12:03:30.583822 22481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:03:30.583902 22481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:03:30.583983 22481 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:03:30.584079 22481 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:03:30.594686 22481 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:39085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:35905
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.190.keytab
--principal=oryx/127.9.221.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:35905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.221.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:03:30.595961 22481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:03:30.597363 22481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:03:30.609431 22489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:30.609781 22493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:30.609946 22490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:03:30.610523 22481 server_base.cc:1034] running on GCE node
I20260811 12:03:30.611290 22481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:03:30.612653 22481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:03:30.613910 22481 hybrid_clock.cc:648] HybridClock initialized: now 1786449810613835 us; error 52 us; skew 500 ppm
I20260811 12:03:30.614230 22481 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:03:30 dist-test-slave-mjjr krb5kdc[22459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449810, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.221.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:03:30.617384 22481 init.cc:377] Logged in from keytab as oryx/127.9.221.190@KRBTEST.COM (short username oryx)
I20260811 12:03:30.618754 22481 webserver.cc:460] Webserver started at http://127.9.221.190:44317/ using document root <none> and password file <none>
I20260811 12:03:30.619436 22481 fs_manager.cc:359] Metadata directory not provided
I20260811 12:03:30.619563 22481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:03:30.619918 22481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:03:30.622750 22481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "bac306678bd5437a8b1101dae6b4b615"
format_stamp: "Formatted at 2026-08-11 12:03:30 on dist-test-slave-mjjr"
server_key: "b4678ef27fac677df94d820895b01118"
server_key_iv: "e50d87f243429ff820647a1637db8c49"
server_key_version: "encryptionkey@0"
I20260811 12:03:30.623569 22481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "bac306678bd5437a8b1101dae6b4b615"
format_stamp: "Formatted at 2026-08-11 12:03:30 on dist-test-slave-mjjr"
server_key: "b4678ef27fac677df94d820895b01118"
server_key_iv: "e50d87f243429ff820647a1637db8c49"
server_key_version: "encryptionkey@0"
I20260811 12:03:30.628585 22481 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260811 12:03:30.631945 22507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:30.633251 22481 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 12:03:30.633440 22481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "bac306678bd5437a8b1101dae6b4b615"
format_stamp: "Formatted at 2026-08-11 12:03:30 on dist-test-slave-mjjr"
server_key: "b4678ef27fac677df94d820895b01118"
server_key_iv: "e50d87f243429ff820647a1637db8c49"
server_key_version: "encryptionkey@0"
I20260811 12:03:30.633654 22481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:03:30.656507 22481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:03:30.660629 22481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:03:30.660907 22481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:03:30.682370 22481 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.190:35905
I20260811 12:03:30.682402 22597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.190:35905 every 8 connection(s)
I20260811 12:03:30.683737 22481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:03:30.687629 22602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:03:30.688582 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 22481
I20260811 12:03:30.688818 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:03:30.689469 10102 external_mini_cluster.cc:1428] Setting key 9e4da4d855864d57d367a822bf9a3b32
I20260811 12:03:30.701845 22602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bac306678bd5437a8b1101dae6b4b615: Bootstrap starting.
Aug 11 12:03:30 dist-test-slave-mjjr krb5kdc[22459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449810, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM
I20260811 12:03:30.705279 22602 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bac306678bd5437a8b1101dae6b4b615: Neither blocks nor log segments found. Creating new log.
I20260811 12:03:30.706272 22602 log.cc:824] T 00000000000000000000000000000000 P bac306678bd5437a8b1101dae6b4b615: Log is configured to *not* fsync() on all Append() calls
I20260811 12:03:30.709427 22602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bac306678bd5437a8b1101dae6b4b615: No bootstrap required, opened a new log
I20260811 12:03:30.713405 22605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:30.691434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33222 (local address 127.9.221.190:35905)
0811 12:03:30.692322 (+ 888us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:30.692336 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:30.692387 (+ 51us) server_negotiation.cc:407] Connection header received
0811 12:03:30.694306 (+ 1919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:30.694334 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:30.694827 (+ 493us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:30.695274 (+ 447us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:30.696627 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:30.698639 (+ 2012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:30.699857 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:30.700872 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:30.701056 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:30.705385 (+ 4329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:30.705398 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:30.705416 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:30.705460 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:30.709089 (+ 3629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:30.709952 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:30.709958 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:30.709964 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:30.710151 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:30.710578 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:30.710582 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:30.710584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:30.711117 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:30.711375 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:30.711924 (+ 549us) server_negotiation.cc:299] Negotiation successful
0811 12:03:30.712361 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":380,"threads_started":1}
I20260811 12:03:30.717648 22602 raft_consensus.cc:348] T 00000000000000000000000000000000 P bac306678bd5437a8b1101dae6b4b615 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bac306678bd5437a8b1101dae6b4b615" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 35905 } }
I20260811 12:03:30.717985 22602 raft_consensus.cc:374] T 00000000000000000000000000000000 P bac306678bd5437a8b1101dae6b4b615 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:03:30.718104 22602 raft_consensus.cc:729] T 00000000000000000000000000000000 P bac306678bd5437a8b1101dae6b4b615 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bac306678bd5437a8b1101dae6b4b615, State: Initialized, Role: FOLLOWER
I20260811 12:03:30.718710 22602 consensus_queue.cc:260] T 00000000000000000000000000000000 P bac306678bd5437a8b1101dae6b4b615 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bac306678bd5437a8b1101dae6b4b615" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 35905 } }
I20260811 12:03:30.718941 22602 raft_consensus.cc:388] T 00000000000000000000000000000000 P bac306678bd5437a8b1101dae6b4b615 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:03:30.719100 22602 raft_consensus.cc:482] T 00000000000000000000000000000000 P bac306678bd5437a8b1101dae6b4b615 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:03:30.719271 22602 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bac306678bd5437a8b1101dae6b4b615 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:03:30.721469 22602 raft_consensus.cc:504] T 00000000000000000000000000000000 P bac306678bd5437a8b1101dae6b4b615 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bac306678bd5437a8b1101dae6b4b615" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 35905 } }
I20260811 12:03:30.722007 22602 leader_election.cc:302] T 00000000000000000000000000000000 P bac306678bd5437a8b1101dae6b4b615 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bac306678bd5437a8b1101dae6b4b615; no voters:
I20260811 12:03:30.722553 22602 leader_election.cc:288] T 00000000000000000000000000000000 P bac306678bd5437a8b1101dae6b4b615 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:03:30.722684 22609 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bac306678bd5437a8b1101dae6b4b615 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:03:30.723387 22609 raft_consensus.cc:686] T 00000000000000000000000000000000 P bac306678bd5437a8b1101dae6b4b615 [term 1 LEADER]: Becoming Leader. State: Replica: bac306678bd5437a8b1101dae6b4b615, State: Running, Role: LEADER
I20260811 12:03:30.724030 22609 consensus_queue.cc:237] T 00000000000000000000000000000000 P bac306678bd5437a8b1101dae6b4b615 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bac306678bd5437a8b1101dae6b4b615" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 35905 } }
I20260811 12:03:30.724699 22602 sys_catalog.cc:564] T 00000000000000000000000000000000 P bac306678bd5437a8b1101dae6b4b615 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:03:30.726624 22610 sys_catalog.cc:455] T 00000000000000000000000000000000 P bac306678bd5437a8b1101dae6b4b615 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bac306678bd5437a8b1101dae6b4b615" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bac306678bd5437a8b1101dae6b4b615" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 35905 } } }
I20260811 12:03:30.727041 22610 sys_catalog.cc:458] T 00000000000000000000000000000000 P bac306678bd5437a8b1101dae6b4b615 [sys.catalog]: This master's current role is: LEADER
I20260811 12:03:30.729444 22611 sys_catalog.cc:455] T 00000000000000000000000000000000 P bac306678bd5437a8b1101dae6b4b615 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bac306678bd5437a8b1101dae6b4b615. Latest consensus state: current_term: 1 leader_uuid: "bac306678bd5437a8b1101dae6b4b615" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bac306678bd5437a8b1101dae6b4b615" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 35905 } } }
I20260811 12:03:30.729702 22611 sys_catalog.cc:458] T 00000000000000000000000000000000 P bac306678bd5437a8b1101dae6b4b615 [sys.catalog]: This master's current role is: LEADER
I20260811 12:03:30.729712 22621 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:03:30.734719 22621 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:03:30.744009 22621 catalog_manager.cc:1347] Generated new cluster ID: 6085b941081648d08aace58f1a7ea29e
I20260811 12:03:30.744128 22621 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:03:30.759702 22621 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:03:30.761155 22621 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:03:30.766736 22621 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bac306678bd5437a8b1101dae6b4b615: Generated new TSK 0
I20260811 12:03:30.767452 22621 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.9.221.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.221.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.221.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.129.keytab.
Entry for principal oryx/127.9.221.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.129.keytab.
I20260811 12:03:30.833371 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.129:0
--local_ip_for_outbound_sockets=127.9.221.129
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:35905
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.129.keytab
--principal=oryx/127.9.221.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:39085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:03:31.185941 22639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:03:31.186329 22639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:03:31.186560 22639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:03:31.195554 22639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:03:31.195744 22639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:03:31.195895 22639 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 12:03:31.196004 22639 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.129
I20260811 12:03:31.207350 22639 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:39085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.129.keytab
--principal=oryx/127.9.221.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.221.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.221.190:35905
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.129
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:03:31.208592 22639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:03:31.210062 22639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:03:31.225961 22645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:31.226686 22648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:31.226907 22646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:03:31.228772 22639 server_base.cc:1034] running on GCE node
I20260811 12:03:31.229367 22639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:03:31.230578 22639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:03:31.231938 22639 hybrid_clock.cc:648] HybridClock initialized: now 1786449811231821 us; error 40 us; skew 500 ppm
I20260811 12:03:31.232225 22639 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:03:31 dist-test-slave-mjjr krb5kdc[22459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449811, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.221.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:03:31.236121 22639 init.cc:377] Logged in from keytab as oryx/127.9.221.129@KRBTEST.COM (short username oryx)
I20260811 12:03:31.237738 22639 webserver.cc:460] Webserver started at http://127.9.221.129:43817/ using document root <none> and password file <none>
I20260811 12:03:31.238492 22639 fs_manager.cc:359] Metadata directory not provided
I20260811 12:03:31.238622 22639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:03:31.238943 22639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:03:31.242669 22639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-0/data/instance:
uuid: "9265c12d3059486a8a74caa87ca611d4"
format_stamp: "Formatted at 2026-08-11 12:03:31 on dist-test-slave-mjjr"
server_key: "ebd64ba59128b8eae792572e09e2008c"
server_key_iv: "31d9cb503546b2746dd86c589896deae"
server_key_version: "encryptionkey@0"
I20260811 12:03:31.243597 22639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance:
uuid: "9265c12d3059486a8a74caa87ca611d4"
format_stamp: "Formatted at 2026-08-11 12:03:31 on dist-test-slave-mjjr"
server_key: "ebd64ba59128b8eae792572e09e2008c"
server_key_iv: "31d9cb503546b2746dd86c589896deae"
server_key_version: "encryptionkey@0"
I20260811 12:03:31.250397 22639 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.004s
I20260811 12:03:31.254534 22656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:31.256052 22639 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:03:31.256181 22639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-0/wal
uuid: "9265c12d3059486a8a74caa87ca611d4"
format_stamp: "Formatted at 2026-08-11 12:03:31 on dist-test-slave-mjjr"
server_key: "ebd64ba59128b8eae792572e09e2008c"
server_key_iv: "31d9cb503546b2746dd86c589896deae"
server_key_version: "encryptionkey@0"
I20260811 12:03:31.256378 22639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:03:31.268846 22639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:03:31.273662 22639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:03:31.274011 22639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:03:31.277648 22639 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:03:31.277827 22639 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:31.278047 22639 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:03:31.278168 22639 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 12:03:31 dist-test-slave-mjjr krb5kdc[22459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449811, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.221.129@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM
I20260811 12:03:31.312139 22665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:31.278479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:57937 (local address 127.9.221.190:35905)
0811 12:03:31.279999 (+ 1520us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:31.280007 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:31.280954 (+ 947us) server_negotiation.cc:407] Connection header received
0811 12:03:31.283918 (+ 2964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:31.283924 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:31.284035 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:31.284174 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:31.286794 (+ 2620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:31.287987 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:31.291811 (+ 3824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:31.292690 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:31.292938 (+ 248us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:31.299050 (+ 6112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:31.299057 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:31.299062 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:31.299107 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:31.303912 (+ 4805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:31.305260 (+ 1348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:31.305269 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:31.305275 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:31.305509 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:31.306361 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:31.306369 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:31.306375 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:31.306872 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:31.310981 (+ 4109us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:31.311216 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 12:03:31.311532 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1291,"thread_start_us":1176,"threads_started":1}
I20260811 12:03:31.312139 22666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:31.279387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:35905 (local address 127.9.221.129:57937)
0811 12:03:31.280782 (+ 1395us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:31.280828 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:31.283528 (+ 2700us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:31.284472 (+ 944us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:31.284480 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:31.285443 (+ 963us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:31.286529 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:31.286557 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:31.288178 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:31.288190 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:31.291575 (+ 3385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:31.291595 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:31.293420 (+ 1825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:31.293428 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:31.293893 (+ 465us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:31.294892 (+ 999us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:31.294925 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:31.298721 (+ 3796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:31.304131 (+ 5410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:31.304144 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:31.304161 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:31.305095 (+ 934us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:31.305686 (+ 591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:31.305694 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:31.305701 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:31.306156 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:31.307074 (+ 918us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:31.307085 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:31.307712 (+ 627us) client_negotiation.cc:769] Sending connection context
0811 12:03:31.308084 (+ 372us) client_negotiation.cc:240] Negotiation successful
0811 12:03:31.308779 (+ 695us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1167,"thread_start_us":839,"threads_started":1}
I20260811 12:03:31.351213 22755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:31.328836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:35905 (local address 127.9.221.129:50835)
0811 12:03:31.330739 (+ 1903us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:31.330822 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:31.331036 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:31.331595 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:31.331602 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:31.333023 (+ 1421us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:31.334771 (+ 1748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:31.334793 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:31.336917 (+ 2124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:31.336943 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:31.340108 (+ 3165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:31.340129 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:31.341327 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:31.341337 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:31.341583 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:31.342389 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:31.342412 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:31.343933 (+ 1521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:31.347814 (+ 3881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:31.347819 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:31.347823 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:31.348466 (+ 643us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:31.348885 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:31.348889 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:31.348893 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:31.349051 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:31.349710 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:31.349715 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:31.350009 (+ 294us) client_negotiation.cc:769] Sending connection context
0811 12:03:31.350190 (+ 181us) client_negotiation.cc:240] Negotiation successful
0811 12:03:31.350500 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1639,"thread_start_us":956,"threads_started":1}
I20260811 12:03:31.351917 22665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:31.328635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:50835 (local address 127.9.221.190:35905)
0811 12:03:31.328880 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:31.328888 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:31.330936 (+ 2048us) server_negotiation.cc:407] Connection header received
0811 12:03:31.331214 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:31.331221 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:31.331396 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:31.331529 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:31.334989 (+ 3460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:31.336700 (+ 1711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:31.340304 (+ 3604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:31.341073 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:31.341229 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:31.344164 (+ 2935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:31.344171 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:31.344176 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:31.344220 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:31.347644 (+ 3424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:31.348610 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:31.348616 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:31.348619 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:31.348750 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:31.349169 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:31.349174 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:31.349177 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:31.349583 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:31.349759 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:31.351290 (+ 1531us) server_negotiation.cc:299] Negotiation successful
0811 12:03:31.351553 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 12:03:31.354486 22639 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.129:38995
I20260811 12:03:31.354511 22827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.129:38995 every 8 connection(s)
I20260811 12:03:31.356077 22639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
I20260811 12:03:31.358068 22828 heartbeater.cc:347] Connected to a master server at 127.9.221.190:35905
I20260811 12:03:31.358314 22828 heartbeater.cc:464] Registering TS with master...
I20260811 12:03:31.358933 22828 heartbeater.cc:511] Master 127.9.221.190:35905 requested a full tablet report, sending...
I20260811 12:03:31.359438 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 22639
I20260811 12:03:31.359714 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance
I20260811 12:03:31.360430 10102 external_mini_cluster.cc:1428] Setting key c1fc618fbb0292c0cdb87d0423c82aa6
I20260811 12:03:31.361090 22535 ts_manager.cc:194] Registered new tserver with Master: 9265c12d3059486a8a74caa87ca611d4 (127.9.221.129:38995)
I20260811 12:03:31.362504 22535 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.221.129@KRBTEST.COM'} at 127.9.221.129:57937
WARNING: no policy specified for oryx/127.9.221.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.221.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.221.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.130.keytab.
Entry for principal oryx/127.9.221.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.130.keytab.
I20260811 12:03:31.410506 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.130:0
--local_ip_for_outbound_sockets=127.9.221.130
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:35905
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.130.keytab
--principal=oryx/127.9.221.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:39085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:03:31.737521 22834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:03:31.737833 22834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:03:31.738010 22834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:03:31.745186 22834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:03:31.745316 22834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:03:31.745465 22834 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 12:03:31.745601 22834 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.130
I20260811 12:03:31.755098 22834 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:39085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.130.keytab
--principal=oryx/127.9.221.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.221.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.221.190:35905
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.130
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:03:31.756376 22834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:03:31.757697 22834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:03:31.771421 22840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:31.771497 22844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:31.771525 22842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:03:31.772291 22834 server_base.cc:1034] running on GCE node
I20260811 12:03:31.772902 22834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:03:31.773741 22834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:03:31.774957 22834 hybrid_clock.cc:648] HybridClock initialized: now 1786449811774879 us; error 47 us; skew 500 ppm
I20260811 12:03:31.775242 22834 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:03:31 dist-test-slave-mjjr krb5kdc[22459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449811, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.221.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:03:31.778470 22834 init.cc:377] Logged in from keytab as oryx/127.9.221.130@KRBTEST.COM (short username oryx)
I20260811 12:03:31.779852 22834 webserver.cc:460] Webserver started at http://127.9.221.130:33475/ using document root <none> and password file <none>
I20260811 12:03:31.780490 22834 fs_manager.cc:359] Metadata directory not provided
I20260811 12:03:31.780577 22834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:03:31.780930 22834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:03:31.783694 22834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-1/data/instance:
uuid: "af341a2adada48adaa44eaa31c48cc05"
format_stamp: "Formatted at 2026-08-11 12:03:31 on dist-test-slave-mjjr"
server_key: "434c0a641ae82e6947739f14cd8965b1"
server_key_iv: "2544a213ad38380b92eb615382ddfa65"
server_key_version: "encryptionkey@0"
I20260811 12:03:31.784447 22834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance:
uuid: "af341a2adada48adaa44eaa31c48cc05"
format_stamp: "Formatted at 2026-08-11 12:03:31 on dist-test-slave-mjjr"
server_key: "434c0a641ae82e6947739f14cd8965b1"
server_key_iv: "2544a213ad38380b92eb615382ddfa65"
server_key_version: "encryptionkey@0"
I20260811 12:03:31.789211 22834 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 12:03:31.792518 22853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:31.793733 22834 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 12:03:31.793905 22834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-1/wal
uuid: "af341a2adada48adaa44eaa31c48cc05"
format_stamp: "Formatted at 2026-08-11 12:03:31 on dist-test-slave-mjjr"
server_key: "434c0a641ae82e6947739f14cd8965b1"
server_key_iv: "2544a213ad38380b92eb615382ddfa65"
server_key_version: "encryptionkey@0"
I20260811 12:03:31.794131 22834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:03:31.814471 22834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:03:31.818904 22834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:03:31.819196 22834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:03:31.822012 22834 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:03:31.822127 22834 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:31.822283 22834 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:03:31.822360 22834 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 12:03:31 dist-test-slave-mjjr krb5kdc[22459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449811, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.221.130@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM
I20260811 12:03:31.846429 22665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:31.822738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:58101 (local address 127.9.221.190:35905)
0811 12:03:31.822944 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:31.822950 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:31.824423 (+ 1473us) server_negotiation.cc:407] Connection header received
0811 12:03:31.826680 (+ 2257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:31.826685 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:31.826774 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:31.826897 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:31.829008 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:31.830071 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:31.832992 (+ 2921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:31.833706 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:31.833837 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:31.838883 (+ 5046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:31.838887 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:31.838890 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:31.838922 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:31.841977 (+ 3055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:31.843189 (+ 1212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:31.843197 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:31.843202 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:31.843363 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:31.844017 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:31.844024 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:31.844029 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:31.844449 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:31.844662 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:31.845544 (+ 882us) server_negotiation.cc:299] Negotiation successful
0811 12:03:31.845850 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 12:03:31.846969 22865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:31.823347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:35905 (local address 127.9.221.130:58101)
0811 12:03:31.824240 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:31.824290 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:31.826384 (+ 2094us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:31.827153 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:31.827166 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:31.828016 (+ 850us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:31.828826 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:31.828848 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:31.830238 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:31.830250 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:31.832839 (+ 2589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:31.832851 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:31.834207 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:31.834217 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:31.834523 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:31.835313 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:31.835342 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:31.838651 (+ 3309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:31.842150 (+ 3499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:31.842160 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:31.842183 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:31.843017 (+ 834us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:31.843524 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:31.843527 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:31.843530 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:31.843843 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:31.844621 (+ 778us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:31.844631 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:31.845043 (+ 412us) client_negotiation.cc:769] Sending connection context
0811 12:03:31.845397 (+ 354us) client_negotiation.cc:240] Negotiation successful
0811 12:03:31.845804 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":391,"threads_started":1}
I20260811 12:03:31.870728 22970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:31.853650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:35905 (local address 127.9.221.130:42687)
0811 12:03:31.854258 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:31.854276 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:31.854432 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:31.854776 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:31.854781 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:31.855225 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:31.855504 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:31.855512 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:31.856676 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:31.856681 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:31.858724 (+ 2043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:31.858744 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:31.859917 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:31.859938 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:31.860069 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:31.860935 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:31.860959 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:31.862413 (+ 1454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:31.866796 (+ 4383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:31.866805 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:31.866811 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:31.867503 (+ 692us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:31.868085 (+ 582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:31.868093 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:31.868099 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:31.868279 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:31.869161 (+ 882us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:31.869169 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:31.869432 (+ 263us) client_negotiation.cc:769] Sending connection context
0811 12:03:31.869615 (+ 183us) client_negotiation.cc:240] Negotiation successful
0811 12:03:31.870081 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":365,"threads_started":1}
I20260811 12:03:31.872113 22665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:31.853483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:42687 (local address 127.9.221.190:35905)
0811 12:03:31.853651 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:31.853655 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:31.854360 (+ 705us) server_negotiation.cc:407] Connection header received
0811 12:03:31.854549 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:31.854554 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:31.854630 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:31.854752 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:31.855668 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:31.856536 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:31.858882 (+ 2346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:31.859452 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:31.859615 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:31.862562 (+ 2947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:31.862567 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:31.862570 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:31.862623 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:31.866616 (+ 3993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:31.867704 (+ 1088us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:31.867712 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:31.867718 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:31.867910 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:31.868410 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:31.868416 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:31.868420 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:31.868881 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:31.869069 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:31.871301 (+ 2232us) server_negotiation.cc:299] Negotiation successful
0811 12:03:31.871589 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 12:03:31.884718 22834 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.130:46333
I20260811 12:03:31.884801 23036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.130:46333 every 8 connection(s)
I20260811 12:03:31.885913 22834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
I20260811 12:03:31.886947 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 22834
I20260811 12:03:31.887182 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance
I20260811 12:03:31.887773 10102 external_mini_cluster.cc:1428] Setting key 6966204e30c204436d59b53ee7a34f9b
I20260811 12:03:31.888247 23037 heartbeater.cc:347] Connected to a master server at 127.9.221.190:35905
I20260811 12:03:31.888542 23037 heartbeater.cc:464] Registering TS with master...
I20260811 12:03:31.889958 23037 heartbeater.cc:511] Master 127.9.221.190:35905 requested a full tablet report, sending...
I20260811 12:03:31.891324 22535 ts_manager.cc:194] Registered new tserver with Master: af341a2adada48adaa44eaa31c48cc05 (127.9.221.130:46333)
I20260811 12:03:31.892601 22535 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.221.130@KRBTEST.COM'} at 127.9.221.130:58101
WARNING: no policy specified for oryx/127.9.221.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.221.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.221.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.131.keytab.
Entry for principal oryx/127.9.221.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.131.keytab.
I20260811 12:03:31.940358 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.131:0
--local_ip_for_outbound_sockets=127.9.221.131
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:35905
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.131.keytab
--principal=oryx/127.9.221.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:39085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:03:32.276702 23041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:03:32.277029 23041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:03:32.277201 23041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:03:32.284412 23041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:03:32.284564 23041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:03:32.284708 23041 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 12:03:32.284858 23041 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.131
I20260811 12:03:32.294512 23041 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:39085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.131.keytab
--principal=oryx/127.9.221.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.221.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.221.190:35905
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.131
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:03:32.295795 23041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:03:32.297219 23041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:03:32.310128 23050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:32.310212 23046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:32.310187 23048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:03:32.311053 23041 server_base.cc:1034] running on GCE node
I20260811 12:03:32.311735 23041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:03:32.312624 23041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:03:32.313891 23041 hybrid_clock.cc:648] HybridClock initialized: now 1786449812313814 us; error 51 us; skew 500 ppm
I20260811 12:03:32.314185 23041 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:03:32 dist-test-slave-mjjr krb5kdc[22459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449812, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.221.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:03:32.317507 23041 init.cc:377] Logged in from keytab as oryx/127.9.221.131@KRBTEST.COM (short username oryx)
I20260811 12:03:32.318832 23041 webserver.cc:460] Webserver started at http://127.9.221.131:43641/ using document root <none> and password file <none>
I20260811 12:03:32.319453 23041 fs_manager.cc:359] Metadata directory not provided
I20260811 12:03:32.319537 23041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:03:32.319873 23041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:03:32.322629 23041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-2/data/instance:
uuid: "ae4c5de7483641ffb0fd3735fd550767"
format_stamp: "Formatted at 2026-08-11 12:03:32 on dist-test-slave-mjjr"
server_key: "a3431aa100de3a9858dcf140b8ab2ddb"
server_key_iv: "d4e3ca85bda01122f5f8a39a05ff19ea"
server_key_version: "encryptionkey@0"
I20260811 12:03:32.323360 23041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance:
uuid: "ae4c5de7483641ffb0fd3735fd550767"
format_stamp: "Formatted at 2026-08-11 12:03:32 on dist-test-slave-mjjr"
server_key: "a3431aa100de3a9858dcf140b8ab2ddb"
server_key_iv: "d4e3ca85bda01122f5f8a39a05ff19ea"
server_key_version: "encryptionkey@0"
I20260811 12:03:32.328198 23041 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260811 12:03:32.331739 23057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:32.332978 23041 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:03:32.333125 23041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-2/wal
uuid: "ae4c5de7483641ffb0fd3735fd550767"
format_stamp: "Formatted at 2026-08-11 12:03:32 on dist-test-slave-mjjr"
server_key: "a3431aa100de3a9858dcf140b8ab2ddb"
server_key_iv: "d4e3ca85bda01122f5f8a39a05ff19ea"
server_key_version: "encryptionkey@0"
I20260811 12:03:32.333344 23041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:03:32.365082 22828 heartbeater.cc:503] Master 127.9.221.190:35905 was elected leader, sending a full tablet report...
I20260811 12:03:32.366282 23041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:03:32.370239 23041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:03:32.370520 23041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:03:32.373225 23041 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:03:32.373343 23041 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:32.373530 23041 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:03:32.373636 23041 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 12:03:32 dist-test-slave-mjjr krb5kdc[22459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449812, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.221.131@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM
I20260811 12:03:32.396706 22665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:32.373603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:51437 (local address 127.9.221.190:35905)
0811 12:03:32.373778 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:32.373792 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:32.375224 (+ 1432us) server_negotiation.cc:407] Connection header received
0811 12:03:32.377353 (+ 2129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:32.377358 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:32.377447 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:32.377553 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:32.379499 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:32.380620 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:32.383211 (+ 2591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:32.383957 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:32.384102 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:32.389018 (+ 4916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:32.389023 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:32.389026 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:32.389058 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:32.391957 (+ 2899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:32.393050 (+ 1093us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:32.393058 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:32.393063 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:32.393238 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:32.393767 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:32.393774 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:32.393778 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:32.394101 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:32.394274 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:32.395885 (+ 1611us) server_negotiation.cc:299] Negotiation successful
0811 12:03:32.396191 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 12:03:32.396855 23071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:32.374261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:35905 (local address 127.9.221.131:51437)
0811 12:03:32.375080 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:32.375119 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:32.377084 (+ 1965us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:32.377773 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:32.377780 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:32.378548 (+ 768us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:32.379302 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:32.379334 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:32.380777 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:32.380784 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:32.383046 (+ 2262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:32.383058 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:32.384236 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:32.384250 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:32.384565 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:32.385366 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:32.385390 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:32.388744 (+ 3354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:32.392140 (+ 3396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:32.392147 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:32.392164 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:32.392899 (+ 735us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:32.393359 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:32.393363 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:32.393365 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:32.393598 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:32.394446 (+ 848us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:32.394454 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:32.394884 (+ 430us) client_negotiation.cc:769] Sending connection context
0811 12:03:32.395250 (+ 366us) client_negotiation.cc:240] Negotiation successful
0811 12:03:32.395853 (+ 603us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":349,"threads_started":1}
I20260811 12:03:32.426762 22665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:32.410386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:33607 (local address 127.9.221.190:35905)
0811 12:03:32.410543 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:32.410547 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:32.411222 (+ 675us) server_negotiation.cc:407] Connection header received
0811 12:03:32.411423 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:32.411427 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:32.411484 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:32.411586 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:32.413109 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:32.414260 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:32.416083 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:32.416846 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:32.417005 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:32.418802 (+ 1797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:32.418811 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:32.418819 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:32.418867 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:32.423189 (+ 4322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:32.424127 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:32.424132 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:32.424136 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:32.424327 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:32.424859 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:32.424865 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:32.424869 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:32.425266 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:32.425418 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:32.426106 (+ 688us) server_negotiation.cc:299] Negotiation successful
0811 12:03:32.426365 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 12:03:32.426918 23205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:32.410526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:35905 (local address 127.9.221.131:33607)
0811 12:03:32.411120 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:32.411136 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:32.411291 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:32.411610 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:32.411615 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:32.412278 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:32.412862 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:32.412883 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:32.414404 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:32.414408 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:32.415893 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:32.415906 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:32.417080 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:32.417104 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:32.417292 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:32.417886 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:32.417903 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:32.418596 (+ 693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:32.423356 (+ 4760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:32.423362 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:32.423368 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:32.423974 (+ 606us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:32.424466 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:32.424474 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:32.424480 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:32.424675 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:32.425394 (+ 719us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:32.425398 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:32.425736 (+ 338us) client_negotiation.cc:769] Sending connection context
0811 12:03:32.425932 (+ 196us) client_negotiation.cc:240] Negotiation successful
0811 12:03:32.426364 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":338,"threads_started":1}
I20260811 12:03:32.428682 23041 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.131:43229
I20260811 12:03:32.428717 23242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.131:43229 every 8 connection(s)
I20260811 12:03:32.430063 23041 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
I20260811 12:03:32.432152 23243 heartbeater.cc:347] Connected to a master server at 127.9.221.190:35905
I20260811 12:03:32.432330 23243 heartbeater.cc:464] Registering TS with master...
I20260811 12:03:32.432879 23243 heartbeater.cc:511] Master 127.9.221.190:35905 requested a full tablet report, sending...
I20260811 12:03:32.434044 22535 ts_manager.cc:194] Registered new tserver with Master: ae4c5de7483641ffb0fd3735fd550767 (127.9.221.131:43229)
I20260811 12:03:32.434891 22535 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.221.131@KRBTEST.COM'} at 127.9.221.131:51437
I20260811 12:03:32.435263 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 23041
I20260811 12:03:32.435457 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance
I20260811 12:03:32.435946 10102 external_mini_cluster.cc:1428] Setting key 8969308b2af410b272f6db6a928107f1
I20260811 12:03:32.438783 10102 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 12:03:32 dist-test-slave-mjjr krb5kdc[22459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449812, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 12:03:32 dist-test-slave-mjjr krb5kdc[22459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:32 dist-test-slave-mjjr krb5kdc[22459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:03:32.894946 23037 heartbeater.cc:503] Master 127.9.221.190:35905 was elected leader, sending a full tablet report...
I20260811 12:03:33.437873 23243 heartbeater.cc:503] Master 127.9.221.190:35905 was elected leader, sending a full tablet report...
W20260811 12:03:33.598127 23254 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.109s user 0.000s sys 0.019s
W20260811 12:03:33.599570 22665 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:32.483049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33224 (local address 127.9.221.190:35905)
0811 12:03:32.483297 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:32.483304 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:32.483423 (+ 119us) server_negotiation.cc:407] Connection header received
0811 12:03:32.483667 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:32.483673 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:32.483759 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:32.483874 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:32.485251 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:32.485997 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:32.487272 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:32.487783 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:32.487921 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:33.599223 (+1111302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33224: BlockingRecv error: recv got EOF from 127.0.0.1:33224 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 11 12:03:33 dist-test-slave-mjjr krb5kdc[22459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449812, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM
I20260811 12:03:33.616475 22665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:33.603286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33240 (local address 127.9.221.190:35905)
0811 12:03:33.603464 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:33.603468 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:33.603831 (+ 363us) server_negotiation.cc:407] Connection header received
0811 12:03:33.604017 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:33.604021 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:33.604111 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:33.604195 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:33.605399 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:33.606131 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:33.607334 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:33.607796 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:33.607888 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:33.611579 (+ 3691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:33.611583 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:33.611586 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:33.611615 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:33.614214 (+ 2599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:33.614781 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:33.614785 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:33.614787 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:33.614909 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:33.615233 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:33.615236 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:33.615238 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:33.615534 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:33.615798 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:33.616012 (+ 214us) server_negotiation.cc:299] Negotiation successful
0811 12:03:33.616221 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 12:03:33.620325 22535 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33240:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:03:33.627403 22535 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:03:33.655714 23274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:33.640269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.130:46333 (local address 127.0.0.1:59724)
0811 12:03:33.641944 (+ 1675us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:33.641976 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:33.642096 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:33.644584 (+ 2488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:33.644594 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:33.644635 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:03:33.645157 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:33.645172 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:33.649342 (+ 4170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:33.649351 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:33.653063 (+ 3712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:33.653079 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:33.654569 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:33.654594 (+ 25us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:33.654777 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:33.654905 (+ 128us) client_negotiation.cc:769] Sending connection context
0811 12:03:33.655034 (+ 129us) client_negotiation.cc:240] Negotiation successful
0811 12:03:33.655222 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1520,"mutex_wait_us":60,"thread_start_us":10202,"threads_started":1}
I20260811 12:03:33.656618 23273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:33.640876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59724 (local address 127.9.221.130:46333)
0811 12:03:33.643807 (+ 2931us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:33.643819 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:33.643856 (+ 37us) server_negotiation.cc:407] Connection header received
0811 12:03:33.643995 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:33.644006 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:33.644288 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:33.645440 (+ 1152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:03:33.645720 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:33.649078 (+ 3358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:33.653232 (+ 4154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:33.654314 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:33.654483 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:33.654732 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:33.655103 (+ 371us) server_negotiation.cc:299] Negotiation successful
0811 12:03:33.656289 (+ 1186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2743,"thread_start_us":2626,"threads_started":1}
I20260811 12:03:33.658226 23270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:33.641373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.131:43229 (local address 127.0.0.1:55462)
0811 12:03:33.650785 (+ 9412us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:33.650806 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:33.650950 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:33.651603 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:33.651608 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:33.651632 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:03:33.652089 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:33.652098 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:33.653540 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:33.653545 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:33.655692 (+ 2147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:33.655714 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:33.657525 (+ 1811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:33.657529 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:33.657677 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:33.657718 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 12:03:33.657807 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 12:03:33.657921 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9250,"thread_start_us":732,"threads_started":1}
I20260811 12:03:33.658329 23276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:33.641905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55462 (local address 127.9.221.131:43229)
0811 12:03:33.650482 (+ 8577us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:33.650493 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:33.651076 (+ 583us) server_negotiation.cc:407] Connection header received
0811 12:03:33.651203 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:33.651210 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:33.651430 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:33.651615 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:03:33.652364 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:33.653390 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:33.655901 (+ 2511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:33.657325 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:33.657474 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:33.657673 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:33.657881 (+ 208us) server_negotiation.cc:299] Negotiation successful
0811 12:03:33.658059 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8382,"thread_start_us":8210,"threads_started":1}
I20260811 12:03:33.659902 23268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:33.639474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:38995 (local address 127.0.0.1:33434)
0811 12:03:33.640990 (+ 1516us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:33.641037 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:33.641237 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:33.651842 (+ 10605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:33.651847 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:33.651872 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:03:33.652373 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:33.652385 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:33.654315 (+ 1930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:33.654320 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:33.656492 (+ 2172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:33.656507 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:33.659073 (+ 2566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:33.659080 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:33.659249 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:33.659289 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 12:03:33.659402 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 12:03:33.659529 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1184,"mutex_wait_us":1,"thread_start_us":1030,"threads_started":1}
I20260811 12:03:33.659920 23269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:33.639627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33434 (local address 127.9.221.129:38995)
0811 12:03:33.651211 (+ 11584us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:33.651220 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:33.651249 (+ 29us) server_negotiation.cc:407] Connection header received
0811 12:03:33.651341 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:33.651348 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:33.651578 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:33.652593 (+ 1015us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:03:33.652806 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:33.654158 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:33.656668 (+ 2510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:33.658861 (+ 2193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:33.659017 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:33.659224 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:33.659472 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 12:03:33.659623 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11387,"thread_start_us":11237,"threads_started":1}
I20260811 12:03:33.660487 22930 tablet_service.cc:1467] Processing CreateTablet for tablet 34006a201eaf472084395633388c0d8d (DEFAULT_TABLE table=test-table [id=f3b589a6b3ce4ac5bc3c94ade40a01cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:03:33.661832 22930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34006a201eaf472084395633388c0d8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:03:33.662406 23142 tablet_service.cc:1467] Processing CreateTablet for tablet 34006a201eaf472084395633388c0d8d (DEFAULT_TABLE table=test-table [id=f3b589a6b3ce4ac5bc3c94ade40a01cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:03:33.663503 23142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34006a201eaf472084395633388c0d8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:03:33.667197 22726 tablet_service.cc:1467] Processing CreateTablet for tablet 34006a201eaf472084395633388c0d8d (DEFAULT_TABLE table=test-table [id=f3b589a6b3ce4ac5bc3c94ade40a01cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:03:33.668514 22726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34006a201eaf472084395633388c0d8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:03:33.672840 23280 tablet_bootstrap.cc:492] T 34006a201eaf472084395633388c0d8d P af341a2adada48adaa44eaa31c48cc05: Bootstrap starting.
I20260811 12:03:33.676002 23280 tablet_bootstrap.cc:654] T 34006a201eaf472084395633388c0d8d P af341a2adada48adaa44eaa31c48cc05: Neither blocks nor log segments found. Creating new log.
I20260811 12:03:33.677009 23281 tablet_bootstrap.cc:492] T 34006a201eaf472084395633388c0d8d P ae4c5de7483641ffb0fd3735fd550767: Bootstrap starting.
I20260811 12:03:33.678186 23280 log.cc:824] T 34006a201eaf472084395633388c0d8d P af341a2adada48adaa44eaa31c48cc05: Log is configured to *not* fsync() on all Append() calls
I20260811 12:03:33.680238 23281 tablet_bootstrap.cc:654] T 34006a201eaf472084395633388c0d8d P ae4c5de7483641ffb0fd3735fd550767: Neither blocks nor log segments found. Creating new log.
I20260811 12:03:33.681419 23280 tablet_bootstrap.cc:492] T 34006a201eaf472084395633388c0d8d P af341a2adada48adaa44eaa31c48cc05: No bootstrap required, opened a new log
I20260811 12:03:33.681458 23281 log.cc:824] T 34006a201eaf472084395633388c0d8d P ae4c5de7483641ffb0fd3735fd550767: Log is configured to *not* fsync() on all Append() calls
I20260811 12:03:33.681734 23280 ts_tablet_manager.cc:1397] T 34006a201eaf472084395633388c0d8d P af341a2adada48adaa44eaa31c48cc05: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 12:03:33.681880 23283 tablet_bootstrap.cc:492] T 34006a201eaf472084395633388c0d8d P 9265c12d3059486a8a74caa87ca611d4: Bootstrap starting.
I20260811 12:03:33.685010 23283 tablet_bootstrap.cc:654] T 34006a201eaf472084395633388c0d8d P 9265c12d3059486a8a74caa87ca611d4: Neither blocks nor log segments found. Creating new log.
I20260811 12:03:33.685921 23283 log.cc:824] T 34006a201eaf472084395633388c0d8d P 9265c12d3059486a8a74caa87ca611d4: Log is configured to *not* fsync() on all Append() calls
I20260811 12:03:33.688195 23281 tablet_bootstrap.cc:492] T 34006a201eaf472084395633388c0d8d P ae4c5de7483641ffb0fd3735fd550767: No bootstrap required, opened a new log
I20260811 12:03:33.688575 23281 ts_tablet_manager.cc:1397] T 34006a201eaf472084395633388c0d8d P ae4c5de7483641ffb0fd3735fd550767: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260811 12:03:33.694212 23280 raft_consensus.cc:348] T 34006a201eaf472084395633388c0d8d P af341a2adada48adaa44eaa31c48cc05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } }
I20260811 12:03:33.694697 23280 raft_consensus.cc:374] T 34006a201eaf472084395633388c0d8d P af341a2adada48adaa44eaa31c48cc05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:03:33.694840 23280 raft_consensus.cc:729] T 34006a201eaf472084395633388c0d8d P af341a2adada48adaa44eaa31c48cc05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af341a2adada48adaa44eaa31c48cc05, State: Initialized, Role: FOLLOWER
I20260811 12:03:33.694875 23283 tablet_bootstrap.cc:492] T 34006a201eaf472084395633388c0d8d P 9265c12d3059486a8a74caa87ca611d4: No bootstrap required, opened a new log
I20260811 12:03:33.695132 23283 ts_tablet_manager.cc:1397] T 34006a201eaf472084395633388c0d8d P 9265c12d3059486a8a74caa87ca611d4: Time spent bootstrapping tablet: real 0.013s user 0.011s sys 0.000s
I20260811 12:03:33.695544 23280 consensus_queue.cc:260] T 34006a201eaf472084395633388c0d8d P af341a2adada48adaa44eaa31c48cc05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } }
I20260811 12:03:33.700134 23281 raft_consensus.cc:348] T 34006a201eaf472084395633388c0d8d P ae4c5de7483641ffb0fd3735fd550767 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } }
I20260811 12:03:33.700579 23281 raft_consensus.cc:374] T 34006a201eaf472084395633388c0d8d P ae4c5de7483641ffb0fd3735fd550767 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:03:33.700709 23281 raft_consensus.cc:729] T 34006a201eaf472084395633388c0d8d P ae4c5de7483641ffb0fd3735fd550767 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae4c5de7483641ffb0fd3735fd550767, State: Initialized, Role: FOLLOWER
I20260811 12:03:33.701326 23281 consensus_queue.cc:260] T 34006a201eaf472084395633388c0d8d P ae4c5de7483641ffb0fd3735fd550767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } }
I20260811 12:03:33.703724 23280 ts_tablet_manager.cc:1428] T 34006a201eaf472084395633388c0d8d P af341a2adada48adaa44eaa31c48cc05: Time spent starting tablet: real 0.022s user 0.019s sys 0.000s
I20260811 12:03:33.707419 23283 raft_consensus.cc:348] T 34006a201eaf472084395633388c0d8d P 9265c12d3059486a8a74caa87ca611d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } }
I20260811 12:03:33.708003 23283 raft_consensus.cc:374] T 34006a201eaf472084395633388c0d8d P 9265c12d3059486a8a74caa87ca611d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:03:33.708124 23283 raft_consensus.cc:729] T 34006a201eaf472084395633388c0d8d P 9265c12d3059486a8a74caa87ca611d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9265c12d3059486a8a74caa87ca611d4, State: Initialized, Role: FOLLOWER
I20260811 12:03:33.708799 23283 consensus_queue.cc:260] T 34006a201eaf472084395633388c0d8d P 9265c12d3059486a8a74caa87ca611d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } }
I20260811 12:03:33.710438 23283 ts_tablet_manager.cc:1428] T 34006a201eaf472084395633388c0d8d P 9265c12d3059486a8a74caa87ca611d4: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260811 12:03:33.711238 23281 ts_tablet_manager.cc:1428] T 34006a201eaf472084395633388c0d8d P ae4c5de7483641ffb0fd3735fd550767: Time spent starting tablet: real 0.023s user 0.012s sys 0.007s
I20260811 12:03:33.755071 23293 raft_consensus.cc:482] T 34006a201eaf472084395633388c0d8d P 9265c12d3059486a8a74caa87ca611d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:03:33.755378 23293 raft_consensus.cc:504] T 34006a201eaf472084395633388c0d8d P 9265c12d3059486a8a74caa87ca611d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } }
I20260811 12:03:33.757045 23293 leader_election.cc:288] T 34006a201eaf472084395633388c0d8d P 9265c12d3059486a8a74caa87ca611d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ae4c5de7483641ffb0fd3735fd550767 (127.9.221.131:43229), af341a2adada48adaa44eaa31c48cc05 (127.9.221.130:46333)
I20260811 12:03:33.764360 23276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:33.757623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:49159 (local address 127.9.221.131:43229)
0811 12:03:33.757833 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:33.757840 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:33.758118 (+ 278us) server_negotiation.cc:407] Connection header received
0811 12:03:33.758278 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:33.758283 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:33.758361 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:33.758467 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:03:33.759168 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:33.760367 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:33.762431 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:33.763323 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:33.763448 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:33.763507 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:33.763950 (+ 443us) server_negotiation.cc:299] Negotiation successful
0811 12:03:33.764080 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 12:03:33.764427 23294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:33.757286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.131:43229 (local address 127.9.221.129:49159)
0811 12:03:33.757988 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:33.758003 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:33.758129 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:33.758496 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:33.758501 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:33.758532 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:03:33.758984 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:33.758999 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:33.760503 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:33.760508 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:33.762247 (+ 1739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:33.762259 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:33.763550 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:33.763556 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:33.763768 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:33.763808 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 12:03:33.763903 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 12:03:33.764038 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"mutex_wait_us":24,"thread_start_us":353,"threads_started":1}
I20260811 12:03:33.764673 23273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:33.757630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:40549 (local address 127.9.221.130:46333)
0811 12:03:33.757856 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:33.757863 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:33.758783 (+ 920us) server_negotiation.cc:407] Connection header received
0811 12:03:33.758926 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:33.758932 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:33.759064 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:33.759154 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:03:33.759904 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:33.760897 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:33.762802 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:33.763698 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:33.763846 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:33.763927 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:33.764292 (+ 365us) server_negotiation.cc:299] Negotiation successful
0811 12:03:33.764445 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 12:03:33.764814 23295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:33.757294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.130:46333 (local address 127.9.221.129:40549)
0811 12:03:33.758651 (+ 1357us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:33.758684 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:33.758800 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:33.759299 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:33.759304 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:33.759330 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:03:33.759735 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:33.759748 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:33.761074 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:33.761078 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:33.762646 (+ 1568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:33.762660 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:33.763922 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:33.763928 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:33.764098 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:33.764132 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 12:03:33.764247 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 12:03:33.764370 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1213,"mutex_wait_us":644,"thread_start_us":398,"threads_started":1}
I20260811 12:03:33.765949 22960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34006a201eaf472084395633388c0d8d" candidate_uuid: "9265c12d3059486a8a74caa87ca611d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af341a2adada48adaa44eaa31c48cc05" is_pre_election: true
I20260811 12:03:33.765949 23172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34006a201eaf472084395633388c0d8d" candidate_uuid: "9265c12d3059486a8a74caa87ca611d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae4c5de7483641ffb0fd3735fd550767" is_pre_election: true
I20260811 12:03:33.766558 23172 raft_consensus.cc:2454] T 34006a201eaf472084395633388c0d8d P ae4c5de7483641ffb0fd3735fd550767 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9265c12d3059486a8a74caa87ca611d4 in term 0.
I20260811 12:03:33.766558 22960 raft_consensus.cc:2454] T 34006a201eaf472084395633388c0d8d P af341a2adada48adaa44eaa31c48cc05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9265c12d3059486a8a74caa87ca611d4 in term 0.
I20260811 12:03:33.767290 22660 leader_election.cc:302] T 34006a201eaf472084395633388c0d8d P 9265c12d3059486a8a74caa87ca611d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 9265c12d3059486a8a74caa87ca611d4, ae4c5de7483641ffb0fd3735fd550767, af341a2adada48adaa44eaa31c48cc05; no voters:
I20260811 12:03:33.767777 23293 raft_consensus.cc:2781] T 34006a201eaf472084395633388c0d8d P 9265c12d3059486a8a74caa87ca611d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:03:33.767925 23293 raft_consensus.cc:482] T 34006a201eaf472084395633388c0d8d P 9265c12d3059486a8a74caa87ca611d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:03:33.768046 23293 raft_consensus.cc:3037] T 34006a201eaf472084395633388c0d8d P 9265c12d3059486a8a74caa87ca611d4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:03:33.770275 23293 raft_consensus.cc:504] T 34006a201eaf472084395633388c0d8d P 9265c12d3059486a8a74caa87ca611d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } }
I20260811 12:03:33.770875 23293 leader_election.cc:288] T 34006a201eaf472084395633388c0d8d P 9265c12d3059486a8a74caa87ca611d4 [CANDIDATE]: Term 1 election: Requested vote from peers ae4c5de7483641ffb0fd3735fd550767 (127.9.221.131:43229), af341a2adada48adaa44eaa31c48cc05 (127.9.221.130:46333)
I20260811 12:03:33.771272 23172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34006a201eaf472084395633388c0d8d" candidate_uuid: "9265c12d3059486a8a74caa87ca611d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae4c5de7483641ffb0fd3735fd550767"
I20260811 12:03:33.771309 22960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34006a201eaf472084395633388c0d8d" candidate_uuid: "9265c12d3059486a8a74caa87ca611d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af341a2adada48adaa44eaa31c48cc05"
I20260811 12:03:33.771467 23172 raft_consensus.cc:3037] T 34006a201eaf472084395633388c0d8d P ae4c5de7483641ffb0fd3735fd550767 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:03:33.771490 22960 raft_consensus.cc:3037] T 34006a201eaf472084395633388c0d8d P af341a2adada48adaa44eaa31c48cc05 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:03:33.773617 23172 raft_consensus.cc:2454] T 34006a201eaf472084395633388c0d8d P ae4c5de7483641ffb0fd3735fd550767 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9265c12d3059486a8a74caa87ca611d4 in term 1.
I20260811 12:03:33.773718 22960 raft_consensus.cc:2454] T 34006a201eaf472084395633388c0d8d P af341a2adada48adaa44eaa31c48cc05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9265c12d3059486a8a74caa87ca611d4 in term 1.
I20260811 12:03:33.773991 22660 leader_election.cc:302] T 34006a201eaf472084395633388c0d8d P 9265c12d3059486a8a74caa87ca611d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9265c12d3059486a8a74caa87ca611d4, ae4c5de7483641ffb0fd3735fd550767; no voters:
I20260811 12:03:33.774290 23293 raft_consensus.cc:2781] T 34006a201eaf472084395633388c0d8d P 9265c12d3059486a8a74caa87ca611d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:03:33.774740 23293 raft_consensus.cc:686] T 34006a201eaf472084395633388c0d8d P 9265c12d3059486a8a74caa87ca611d4 [term 1 LEADER]: Becoming Leader. State: Replica: 9265c12d3059486a8a74caa87ca611d4, State: Running, Role: LEADER
I20260811 12:03:33.775187 23293 consensus_queue.cc:237] T 34006a201eaf472084395633388c0d8d P 9265c12d3059486a8a74caa87ca611d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } }
I20260811 12:03:33.779343 22533 catalog_manager.cc:5495] T 34006a201eaf472084395633388c0d8d P 9265c12d3059486a8a74caa87ca611d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9265c12d3059486a8a74caa87ca611d4 (127.9.221.129). New cstate: current_term: 1 leader_uuid: "9265c12d3059486a8a74caa87ca611d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } health_report { overall_health: UNKNOWN } } }
I20260811 12:03:33.801118 23269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:33.795823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33450 (local address 127.9.221.129:38995)
0811 12:03:33.795998 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:33.796002 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:33.796019 (+ 17us) server_negotiation.cc:407] Connection header received
0811 12:03:33.796198 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:33.796202 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:33.796291 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:33.796407 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:03:33.796722 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:33.797496 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:33.799004 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:33.799377 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:33.799479 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:33.800011 (+ 532us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:03:33.800583 (+ 572us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:03:33.800687 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:33.800767 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 12:03:33.800826 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 12:03:33.808705 22960 raft_consensus.cc:1261] T 34006a201eaf472084395633388c0d8d P af341a2adada48adaa44eaa31c48cc05 [term 1 FOLLOWER]: Refusing update from remote peer 9265c12d3059486a8a74caa87ca611d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:03:33.808727 23172 raft_consensus.cc:1261] T 34006a201eaf472084395633388c0d8d P ae4c5de7483641ffb0fd3735fd550767 [term 1 FOLLOWER]: Refusing update from remote peer 9265c12d3059486a8a74caa87ca611d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:03:33.809690 23296 consensus_queue.cc:1035] T 34006a201eaf472084395633388c0d8d P 9265c12d3059486a8a74caa87ca611d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:03:33.810022 23293 consensus_queue.cc:1035] T 34006a201eaf472084395633388c0d8d P 9265c12d3059486a8a74caa87ca611d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:03:33.822399 23300 mvcc.cc:205] Tried to move back new op lower bound from 7317298437351489536 to 7317298437226602496. Current Snapshot: MvccSnapshot[applied={T|T < 7317298437351489536}]
I20260811 12:03:33.823479 23304 mvcc.cc:205] Tried to move back new op lower bound from 7317298437351489536 to 7317298437226602496. Current Snapshot: MvccSnapshot[applied={T|T < 7317298437351489536}]
I20260811 12:03:33.835278 22535 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:03:33.835549 22535 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:03:33.839174 22535 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bac306678bd5437a8b1101dae6b4b615: Sending DeleteTablet for 3 replicas of tablet 34006a201eaf472084395633388c0d8d
I20260811 12:03:33.840296 22726 tablet_service.cc:1514] Processing DeleteTablet for tablet 34006a201eaf472084395633388c0d8d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:03:33 UTC) from {username='oryx', principal='oryx/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:33434
I20260811 12:03:33.840330 22930 tablet_service.cc:1514] Processing DeleteTablet for tablet 34006a201eaf472084395633388c0d8d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:03:33 UTC) from {username='oryx', principal='oryx/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:59724
I20260811 12:03:33.840582 23142 tablet_service.cc:1514] Processing DeleteTablet for tablet 34006a201eaf472084395633388c0d8d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:03:33 UTC) from {username='oryx', principal='oryx/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:55462
I20260811 12:03:33.842177 22535 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33240:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:03:33.843238 22535 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:03:33.845078 23315 tablet_replica.cc:330] T 34006a201eaf472084395633388c0d8d P af341a2adada48adaa44eaa31c48cc05: stopping tablet replica
I20260811 12:03:33.845572 23315 raft_consensus.cc:2229] T 34006a201eaf472084395633388c0d8d P af341a2adada48adaa44eaa31c48cc05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:03:33.846153 23315 raft_consensus.cc:2258] T 34006a201eaf472084395633388c0d8d P af341a2adada48adaa44eaa31c48cc05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:03:33.847004 23316 tablet_replica.cc:330] T 34006a201eaf472084395633388c0d8d P ae4c5de7483641ffb0fd3735fd550767: stopping tablet replica
I20260811 12:03:33.847407 23316 raft_consensus.cc:2229] T 34006a201eaf472084395633388c0d8d P ae4c5de7483641ffb0fd3735fd550767 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:03:33.847898 23316 raft_consensus.cc:2258] T 34006a201eaf472084395633388c0d8d P ae4c5de7483641ffb0fd3735fd550767 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:03:33.849823 23315 ts_tablet_manager.cc:1905] T 34006a201eaf472084395633388c0d8d P af341a2adada48adaa44eaa31c48cc05: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:03:33.853528 22725 tablet_service.cc:1467] Processing CreateTablet for tablet 46f67bffd7364bbc89237072d52ec9fa (DEFAULT_TABLE table=test-table [id=a1296519484b48e49830f76aab9f522e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:03:33.854184 22725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46f67bffd7364bbc89237072d52ec9fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:03:33.857244 22929 tablet_service.cc:1467] Processing CreateTablet for tablet 46f67bffd7364bbc89237072d52ec9fa (DEFAULT_TABLE table=test-table [id=a1296519484b48e49830f76aab9f522e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:03:33.857707 22929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46f67bffd7364bbc89237072d52ec9fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:03:33.858235 23316 ts_tablet_manager.cc:1905] T 34006a201eaf472084395633388c0d8d P ae4c5de7483641ffb0fd3735fd550767: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:03:33.861264 23142 tablet_service.cc:1467] Processing CreateTablet for tablet 46f67bffd7364bbc89237072d52ec9fa (DEFAULT_TABLE table=test-table [id=a1296519484b48e49830f76aab9f522e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:03:33.861680 23142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46f67bffd7364bbc89237072d52ec9fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:03:33.864604 23315 ts_tablet_manager.cc:1918] T 34006a201eaf472084395633388c0d8d P af341a2adada48adaa44eaa31c48cc05: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:03:33.864825 23315 log.cc:1196] T 34006a201eaf472084395633388c0d8d P af341a2adada48adaa44eaa31c48cc05: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-1/wal/wals/34006a201eaf472084395633388c0d8d
I20260811 12:03:33.865495 23315 ts_tablet_manager.cc:1939] T 34006a201eaf472084395633388c0d8d P af341a2adada48adaa44eaa31c48cc05: Deleting consensus metadata
I20260811 12:03:33.867499 22513 catalog_manager.cc:4841] TS af341a2adada48adaa44eaa31c48cc05 (127.9.221.130:46333): tablet 34006a201eaf472084395633388c0d8d (table test-table [id=f3b589a6b3ce4ac5bc3c94ade40a01cc]) successfully deleted
I20260811 12:03:33.877327 23281 tablet_bootstrap.cc:492] T 46f67bffd7364bbc89237072d52ec9fa P ae4c5de7483641ffb0fd3735fd550767: Bootstrap starting.
I20260811 12:03:33.879112 23316 ts_tablet_manager.cc:1918] T 34006a201eaf472084395633388c0d8d P ae4c5de7483641ffb0fd3735fd550767: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:03:33.879273 23316 log.cc:1196] T 34006a201eaf472084395633388c0d8d P ae4c5de7483641ffb0fd3735fd550767: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-2/wal/wals/34006a201eaf472084395633388c0d8d
I20260811 12:03:33.879761 23281 tablet_bootstrap.cc:654] T 46f67bffd7364bbc89237072d52ec9fa P ae4c5de7483641ffb0fd3735fd550767: Neither blocks nor log segments found. Creating new log.
I20260811 12:03:33.879987 23316 ts_tablet_manager.cc:1939] T 34006a201eaf472084395633388c0d8d P ae4c5de7483641ffb0fd3735fd550767: Deleting consensus metadata
I20260811 12:03:33.881873 22515 catalog_manager.cc:4841] TS ae4c5de7483641ffb0fd3735fd550767 (127.9.221.131:43229): tablet 34006a201eaf472084395633388c0d8d (table test-table [id=f3b589a6b3ce4ac5bc3c94ade40a01cc]) successfully deleted
I20260811 12:03:33.882827 23280 tablet_bootstrap.cc:492] T 46f67bffd7364bbc89237072d52ec9fa P af341a2adada48adaa44eaa31c48cc05: Bootstrap starting.
I20260811 12:03:33.884397 23280 tablet_bootstrap.cc:654] T 46f67bffd7364bbc89237072d52ec9fa P af341a2adada48adaa44eaa31c48cc05: Neither blocks nor log segments found. Creating new log.
I20260811 12:03:33.886381 23283 tablet_bootstrap.cc:492] T 46f67bffd7364bbc89237072d52ec9fa P 9265c12d3059486a8a74caa87ca611d4: Bootstrap starting.
I20260811 12:03:33.887104 23314 tablet_replica.cc:330] T 34006a201eaf472084395633388c0d8d P 9265c12d3059486a8a74caa87ca611d4: stopping tablet replica
I20260811 12:03:33.887584 23314 raft_consensus.cc:2229] T 34006a201eaf472084395633388c0d8d P 9265c12d3059486a8a74caa87ca611d4 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:03:33.888132 23283 tablet_bootstrap.cc:654] T 46f67bffd7364bbc89237072d52ec9fa P 9265c12d3059486a8a74caa87ca611d4: Neither blocks nor log segments found. Creating new log.
I20260811 12:03:33.888288 23314 raft_consensus.cc:2258] T 34006a201eaf472084395633388c0d8d P 9265c12d3059486a8a74caa87ca611d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:03:33.890066 23283 tablet_bootstrap.cc:492] T 46f67bffd7364bbc89237072d52ec9fa P 9265c12d3059486a8a74caa87ca611d4: No bootstrap required, opened a new log
I20260811 12:03:33.890182 23314 ts_tablet_manager.cc:1905] T 34006a201eaf472084395633388c0d8d P 9265c12d3059486a8a74caa87ca611d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:03:33.890303 23283 ts_tablet_manager.cc:1397] T 46f67bffd7364bbc89237072d52ec9fa P 9265c12d3059486a8a74caa87ca611d4: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 12:03:33.891140 23283 raft_consensus.cc:348] T 46f67bffd7364bbc89237072d52ec9fa P 9265c12d3059486a8a74caa87ca611d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } }
I20260811 12:03:33.891472 23283 raft_consensus.cc:374] T 46f67bffd7364bbc89237072d52ec9fa P 9265c12d3059486a8a74caa87ca611d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:03:33.891695 23283 raft_consensus.cc:729] T 46f67bffd7364bbc89237072d52ec9fa P 9265c12d3059486a8a74caa87ca611d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9265c12d3059486a8a74caa87ca611d4, State: Initialized, Role: FOLLOWER
I20260811 12:03:33.892068 23283 consensus_queue.cc:260] T 46f67bffd7364bbc89237072d52ec9fa P 9265c12d3059486a8a74caa87ca611d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } }
I20260811 12:03:33.892802 23281 tablet_bootstrap.cc:492] T 46f67bffd7364bbc89237072d52ec9fa P ae4c5de7483641ffb0fd3735fd550767: No bootstrap required, opened a new log
I20260811 12:03:33.892997 23281 ts_tablet_manager.cc:1397] T 46f67bffd7364bbc89237072d52ec9fa P ae4c5de7483641ffb0fd3735fd550767: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.001s
I20260811 12:03:33.893829 23281 raft_consensus.cc:348] T 46f67bffd7364bbc89237072d52ec9fa P ae4c5de7483641ffb0fd3735fd550767 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } }
I20260811 12:03:33.894122 23281 raft_consensus.cc:374] T 46f67bffd7364bbc89237072d52ec9fa P ae4c5de7483641ffb0fd3735fd550767 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:03:33.894282 23281 raft_consensus.cc:729] T 46f67bffd7364bbc89237072d52ec9fa P ae4c5de7483641ffb0fd3735fd550767 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae4c5de7483641ffb0fd3735fd550767, State: Initialized, Role: FOLLOWER
I20260811 12:03:33.894657 23281 consensus_queue.cc:260] T 46f67bffd7364bbc89237072d52ec9fa P ae4c5de7483641ffb0fd3735fd550767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } }
I20260811 12:03:33.895210 23283 ts_tablet_manager.cc:1428] T 46f67bffd7364bbc89237072d52ec9fa P 9265c12d3059486a8a74caa87ca611d4: Time spent starting tablet: real 0.005s user 0.000s sys 0.003s
I20260811 12:03:33.897208 23314 ts_tablet_manager.cc:1918] T 34006a201eaf472084395633388c0d8d P 9265c12d3059486a8a74caa87ca611d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:03:33.897423 23314 log.cc:1196] T 34006a201eaf472084395633388c0d8d P 9265c12d3059486a8a74caa87ca611d4: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-0/wal/wals/34006a201eaf472084395633388c0d8d
I20260811 12:03:33.897503 23281 ts_tablet_manager.cc:1428] T 46f67bffd7364bbc89237072d52ec9fa P ae4c5de7483641ffb0fd3735fd550767: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260811 12:03:33.898133 23314 ts_tablet_manager.cc:1939] T 34006a201eaf472084395633388c0d8d P 9265c12d3059486a8a74caa87ca611d4: Deleting consensus metadata
I20260811 12:03:33.898512 23280 tablet_bootstrap.cc:492] T 46f67bffd7364bbc89237072d52ec9fa P af341a2adada48adaa44eaa31c48cc05: No bootstrap required, opened a new log
I20260811 12:03:33.898703 23280 ts_tablet_manager.cc:1397] T 46f67bffd7364bbc89237072d52ec9fa P af341a2adada48adaa44eaa31c48cc05: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.003s
I20260811 12:03:33.899610 23280 raft_consensus.cc:348] T 46f67bffd7364bbc89237072d52ec9fa P af341a2adada48adaa44eaa31c48cc05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } }
I20260811 12:03:33.899910 23280 raft_consensus.cc:374] T 46f67bffd7364bbc89237072d52ec9fa P af341a2adada48adaa44eaa31c48cc05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:03:33.900043 23280 raft_consensus.cc:729] T 46f67bffd7364bbc89237072d52ec9fa P af341a2adada48adaa44eaa31c48cc05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af341a2adada48adaa44eaa31c48cc05, State: Initialized, Role: FOLLOWER
I20260811 12:03:33.900177 22515 catalog_manager.cc:4841] TS 9265c12d3059486a8a74caa87ca611d4 (127.9.221.129:38995): tablet 34006a201eaf472084395633388c0d8d (table test-table [id=f3b589a6b3ce4ac5bc3c94ade40a01cc]) successfully deleted
I20260811 12:03:33.900434 23280 consensus_queue.cc:260] T 46f67bffd7364bbc89237072d52ec9fa P af341a2adada48adaa44eaa31c48cc05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } }
I20260811 12:03:33.901085 23280 ts_tablet_manager.cc:1428] T 46f67bffd7364bbc89237072d52ec9fa P af341a2adada48adaa44eaa31c48cc05: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
W20260811 12:03:33.932915 23244 tablet.cc:2369] T 46f67bffd7364bbc89237072d52ec9fa P ae4c5de7483641ffb0fd3735fd550767: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:03:33.996165 23288 raft_consensus.cc:482] T 46f67bffd7364bbc89237072d52ec9fa P ae4c5de7483641ffb0fd3735fd550767 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:03:33.996412 23288 raft_consensus.cc:504] T 46f67bffd7364bbc89237072d52ec9fa P ae4c5de7483641ffb0fd3735fd550767 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } }
I20260811 12:03:33.998466 23288 leader_election.cc:288] T 46f67bffd7364bbc89237072d52ec9fa P ae4c5de7483641ffb0fd3735fd550767 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9265c12d3059486a8a74caa87ca611d4 (127.9.221.129:38995), af341a2adada48adaa44eaa31c48cc05 (127.9.221.130:46333)
I20260811 12:03:34.006399 23269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:33.998821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:57447 (local address 127.9.221.129:38995)
0811 12:03:33.999001 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:33.999009 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:33.999532 (+ 523us) server_negotiation.cc:407] Connection header received
0811 12:03:33.999853 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:33.999859 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:33.999968 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:34.000045 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:03:34.001081 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:34.002287 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:34.004288 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:34.005306 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:34.005457 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:34.005527 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:34.005886 (+ 359us) server_negotiation.cc:299] Negotiation successful
0811 12:03:34.006022 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 12:03:34.006418 23321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:33.998710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:38995 (local address 127.9.221.131:57447)
0811 12:03:33.999421 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:33.999441 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:33.999668 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:34.000311 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:34.000316 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:34.000348 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:03:34.000861 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:34.000874 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:34.002442 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:34.002448 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:34.004109 (+ 1661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:34.004121 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:34.005516 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:34.005523 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:34.005702 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:34.005738 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 12:03:34.005837 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 12:03:34.005979 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"mutex_wait_us":88,"thread_start_us":385,"threads_started":1}
I20260811 12:03:34.007486 23273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:33.999613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:41133 (local address 127.9.221.130:46333)
0811 12:03:33.999826 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:33.999832 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:34.000471 (+ 639us) server_negotiation.cc:407] Connection header received
0811 12:03:34.000651 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:34.000656 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:34.000757 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:34.000854 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:03:34.001419 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:34.002304 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:34.004940 (+ 2636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:34.006116 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:34.006275 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:34.006340 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:34.007062 (+ 722us) server_negotiation.cc:299] Negotiation successful
0811 12:03:34.007210 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 12:03:34.007534 23322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:33.999101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.130:46333 (local address 127.9.221.131:41133)
0811 12:03:34.000358 (+ 1257us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:34.000374 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:34.000510 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:34.000881 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:34.000885 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:34.000916 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:03:34.001242 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:34.001250 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:34.002477 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:34.002483 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:34.004771 (+ 2288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:34.004787 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:34.006687 (+ 1900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:34.006698 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:34.006881 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:34.006912 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 12:03:34.007011 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 12:03:34.007144 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1133,"mutex_wait_us":1,"thread_start_us":903,"threads_started":1}
I20260811 12:03:34.007418 22763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46f67bffd7364bbc89237072d52ec9fa" candidate_uuid: "ae4c5de7483641ffb0fd3735fd550767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9265c12d3059486a8a74caa87ca611d4" is_pre_election: true
I20260811 12:03:34.008217 22960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46f67bffd7364bbc89237072d52ec9fa" candidate_uuid: "ae4c5de7483641ffb0fd3735fd550767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af341a2adada48adaa44eaa31c48cc05" is_pre_election: true
I20260811 12:03:34.008248 22763 raft_consensus.cc:2454] T 46f67bffd7364bbc89237072d52ec9fa P 9265c12d3059486a8a74caa87ca611d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae4c5de7483641ffb0fd3735fd550767 in term 0.
I20260811 12:03:34.008502 22960 raft_consensus.cc:2454] T 46f67bffd7364bbc89237072d52ec9fa P af341a2adada48adaa44eaa31c48cc05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae4c5de7483641ffb0fd3735fd550767 in term 0.
I20260811 12:03:34.008906 23064 leader_election.cc:302] T 46f67bffd7364bbc89237072d52ec9fa P ae4c5de7483641ffb0fd3735fd550767 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9265c12d3059486a8a74caa87ca611d4, ae4c5de7483641ffb0fd3735fd550767; no voters:
I20260811 12:03:34.009322 23288 raft_consensus.cc:2781] T 46f67bffd7364bbc89237072d52ec9fa P ae4c5de7483641ffb0fd3735fd550767 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:03:34.009512 23288 raft_consensus.cc:482] T 46f67bffd7364bbc89237072d52ec9fa P ae4c5de7483641ffb0fd3735fd550767 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:03:34.009655 23288 raft_consensus.cc:3037] T 46f67bffd7364bbc89237072d52ec9fa P ae4c5de7483641ffb0fd3735fd550767 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:03:34.012961 23288 raft_consensus.cc:504] T 46f67bffd7364bbc89237072d52ec9fa P ae4c5de7483641ffb0fd3735fd550767 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } }
I20260811 12:03:34.013644 23288 leader_election.cc:288] T 46f67bffd7364bbc89237072d52ec9fa P ae4c5de7483641ffb0fd3735fd550767 [CANDIDATE]: Term 1 election: Requested vote from peers 9265c12d3059486a8a74caa87ca611d4 (127.9.221.129:38995), af341a2adada48adaa44eaa31c48cc05 (127.9.221.130:46333)
I20260811 12:03:34.014082 22960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46f67bffd7364bbc89237072d52ec9fa" candidate_uuid: "ae4c5de7483641ffb0fd3735fd550767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af341a2adada48adaa44eaa31c48cc05"
I20260811 12:03:34.014119 22763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46f67bffd7364bbc89237072d52ec9fa" candidate_uuid: "ae4c5de7483641ffb0fd3735fd550767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9265c12d3059486a8a74caa87ca611d4"
I20260811 12:03:34.014295 22960 raft_consensus.cc:3037] T 46f67bffd7364bbc89237072d52ec9fa P af341a2adada48adaa44eaa31c48cc05 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:03:34.014320 22763 raft_consensus.cc:3037] T 46f67bffd7364bbc89237072d52ec9fa P 9265c12d3059486a8a74caa87ca611d4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:03:34.016595 22960 raft_consensus.cc:2454] T 46f67bffd7364bbc89237072d52ec9fa P af341a2adada48adaa44eaa31c48cc05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae4c5de7483641ffb0fd3735fd550767 in term 1.
I20260811 12:03:34.017021 23063 leader_election.cc:302] T 46f67bffd7364bbc89237072d52ec9fa P ae4c5de7483641ffb0fd3735fd550767 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae4c5de7483641ffb0fd3735fd550767, af341a2adada48adaa44eaa31c48cc05; no voters:
I20260811 12:03:34.017292 23288 raft_consensus.cc:2781] T 46f67bffd7364bbc89237072d52ec9fa P ae4c5de7483641ffb0fd3735fd550767 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:03:34.017691 22763 raft_consensus.cc:2454] T 46f67bffd7364bbc89237072d52ec9fa P 9265c12d3059486a8a74caa87ca611d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae4c5de7483641ffb0fd3735fd550767 in term 1.
I20260811 12:03:34.017807 23288 raft_consensus.cc:686] T 46f67bffd7364bbc89237072d52ec9fa P ae4c5de7483641ffb0fd3735fd550767 [term 1 LEADER]: Becoming Leader. State: Replica: ae4c5de7483641ffb0fd3735fd550767, State: Running, Role: LEADER
I20260811 12:03:34.018389 23288 consensus_queue.cc:237] T 46f67bffd7364bbc89237072d52ec9fa P ae4c5de7483641ffb0fd3735fd550767 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } }
I20260811 12:03:34.021938 22533 catalog_manager.cc:5495] T 46f67bffd7364bbc89237072d52ec9fa P ae4c5de7483641ffb0fd3735fd550767 reported cstate change: term changed from 0 to 1, leader changed from <none> to ae4c5de7483641ffb0fd3735fd550767 (127.9.221.131). New cstate: current_term: 1 leader_uuid: "ae4c5de7483641ffb0fd3735fd550767" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } health_report { overall_health: HEALTHY } } }
Aug 11 12:03:34 dist-test-slave-mjjr krb5kdc[22459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449810, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.221.190@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM
I20260811 12:03:34.102027 23332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:34.086082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:35905 (local address 127.0.0.1:33250)
0811 12:03:34.086794 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:34.086819 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:34.086986 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:34.087317 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:34.087323 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:34.088328 (+ 1005us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:34.088712 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:34.088725 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:34.089774 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:34.089779 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:34.091400 (+ 1621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:34.091417 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:34.092175 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:34.092183 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:34.092369 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:34.093038 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:34.093069 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:34.095625 (+ 2556us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:34.099473 (+ 3848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:34.099483 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:34.099488 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:34.100071 (+ 583us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:34.100450 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:34.100453 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:34.100455 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:34.100598 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:34.101206 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:34.101212 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:34.101350 (+ 138us) client_negotiation.cc:769] Sending connection context
0811 12:03:34.101486 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 12:03:34.101660 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":359,"threads_started":1}
I20260811 12:03:34.102296 22665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:34.086090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33250 (local address 127.9.221.190:35905)
0811 12:03:34.086375 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:34.086384 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:34.086913 (+ 529us) server_negotiation.cc:407] Connection header received
0811 12:03:34.087103 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:34.087107 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:34.087164 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:34.087247 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:34.088853 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:34.089623 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:34.091538 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:34.091982 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:34.092073 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:34.095871 (+ 3798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:34.095879 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:34.095884 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:34.095927 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:34.099345 (+ 3418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:34.100178 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:34.100183 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:34.100187 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:34.100328 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:34.100713 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:34.100718 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:34.100722 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:34.101068 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:34.101230 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:34.101643 (+ 413us) server_negotiation.cc:299] Negotiation successful
0811 12:03:34.101864 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124}
I20260811 12:03:34.107528 22533 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:33250:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 12:03:34.108572 22533 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 12:03:34.111218 22829 tablet.cc:2369] T 46f67bffd7364bbc89237072d52ec9fa P 9265c12d3059486a8a74caa87ca611d4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:03:34.115758 22726 tablet_service.cc:1467] Processing CreateTablet for tablet 9ffc36515643440d8e316a877d6586d7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0baff36a738c415d886d9711918caf61]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 12:03:34.115854 22929 tablet_service.cc:1467] Processing CreateTablet for tablet 9ffc36515643440d8e316a877d6586d7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0baff36a738c415d886d9711918caf61]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 12:03:34.116369 22929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ffc36515643440d8e316a877d6586d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:03:34.116396 22726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ffc36515643440d8e316a877d6586d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:03:34.119611 23142 tablet_service.cc:1467] Processing CreateTablet for tablet 9ffc36515643440d8e316a877d6586d7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0baff36a738c415d886d9711918caf61]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 12:03:34.120188 23142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ffc36515643440d8e316a877d6586d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:03:34.123420 23280 tablet_bootstrap.cc:492] T 9ffc36515643440d8e316a877d6586d7 P af341a2adada48adaa44eaa31c48cc05: Bootstrap starting.
I20260811 12:03:34.124923 23283 tablet_bootstrap.cc:492] T 9ffc36515643440d8e316a877d6586d7 P 9265c12d3059486a8a74caa87ca611d4: Bootstrap starting.
I20260811 12:03:34.124909 23280 tablet_bootstrap.cc:654] T 9ffc36515643440d8e316a877d6586d7 P af341a2adada48adaa44eaa31c48cc05: Neither blocks nor log segments found. Creating new log.
I20260811 12:03:34.127027 23283 tablet_bootstrap.cc:654] T 9ffc36515643440d8e316a877d6586d7 P 9265c12d3059486a8a74caa87ca611d4: Neither blocks nor log segments found. Creating new log.
I20260811 12:03:34.130213 23281 tablet_bootstrap.cc:492] T 9ffc36515643440d8e316a877d6586d7 P ae4c5de7483641ffb0fd3735fd550767: Bootstrap starting.
I20260811 12:03:34.132244 23281 tablet_bootstrap.cc:654] T 9ffc36515643440d8e316a877d6586d7 P ae4c5de7483641ffb0fd3735fd550767: Neither blocks nor log segments found. Creating new log.
I20260811 12:03:34.132836 23280 tablet_bootstrap.cc:492] T 9ffc36515643440d8e316a877d6586d7 P af341a2adada48adaa44eaa31c48cc05: No bootstrap required, opened a new log
I20260811 12:03:34.133018 23280 ts_tablet_manager.cc:1397] T 9ffc36515643440d8e316a877d6586d7 P af341a2adada48adaa44eaa31c48cc05: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.005s
I20260811 12:03:34.133679 23283 tablet_bootstrap.cc:492] T 9ffc36515643440d8e316a877d6586d7 P 9265c12d3059486a8a74caa87ca611d4: No bootstrap required, opened a new log
I20260811 12:03:34.133925 23283 ts_tablet_manager.cc:1397] T 9ffc36515643440d8e316a877d6586d7 P 9265c12d3059486a8a74caa87ca611d4: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.001s
I20260811 12:03:34.133996 23280 raft_consensus.cc:348] T 9ffc36515643440d8e316a877d6586d7 P af341a2adada48adaa44eaa31c48cc05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } }
I20260811 12:03:34.134323 23280 raft_consensus.cc:374] T 9ffc36515643440d8e316a877d6586d7 P af341a2adada48adaa44eaa31c48cc05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:03:34.134424 23280 raft_consensus.cc:729] T 9ffc36515643440d8e316a877d6586d7 P af341a2adada48adaa44eaa31c48cc05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af341a2adada48adaa44eaa31c48cc05, State: Initialized, Role: FOLLOWER
I20260811 12:03:34.134721 23280 consensus_queue.cc:260] T 9ffc36515643440d8e316a877d6586d7 P af341a2adada48adaa44eaa31c48cc05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } }
I20260811 12:03:34.135463 23280 ts_tablet_manager.cc:1428] T 9ffc36515643440d8e316a877d6586d7 P af341a2adada48adaa44eaa31c48cc05: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260811 12:03:34.136317 23283 raft_consensus.cc:348] T 9ffc36515643440d8e316a877d6586d7 P 9265c12d3059486a8a74caa87ca611d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } }
I20260811 12:03:34.136756 23283 raft_consensus.cc:374] T 9ffc36515643440d8e316a877d6586d7 P 9265c12d3059486a8a74caa87ca611d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:03:34.136968 23283 raft_consensus.cc:729] T 9ffc36515643440d8e316a877d6586d7 P 9265c12d3059486a8a74caa87ca611d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9265c12d3059486a8a74caa87ca611d4, State: Initialized, Role: FOLLOWER
I20260811 12:03:34.136675 23286 tablet_replica.cc:439] T 9ffc36515643440d8e316a877d6586d7 P af341a2adada48adaa44eaa31c48cc05: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } } }
I20260811 12:03:34.137140 23286 tablet_replica.cc:442] T 9ffc36515643440d8e316a877d6586d7 P af341a2adada48adaa44eaa31c48cc05: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:03:34.137595 23283 consensus_queue.cc:260] T 9ffc36515643440d8e316a877d6586d7 P 9265c12d3059486a8a74caa87ca611d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } }
I20260811 12:03:34.138309 23283 ts_tablet_manager.cc:1428] T 9ffc36515643440d8e316a877d6586d7 P 9265c12d3059486a8a74caa87ca611d4: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260811 12:03:34.139243 23296 tablet_replica.cc:439] T 9ffc36515643440d8e316a877d6586d7 P 9265c12d3059486a8a74caa87ca611d4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } } }
I20260811 12:03:34.139590 23296 tablet_replica.cc:442] T 9ffc36515643440d8e316a877d6586d7 P 9265c12d3059486a8a74caa87ca611d4: This TxnStatusTablet replica's current role is: FOLLOWER
W20260811 12:03:34.141165 23038 tablet.cc:2369] T 46f67bffd7364bbc89237072d52ec9fa P af341a2adada48adaa44eaa31c48cc05: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:03:34.142809 23281 tablet_bootstrap.cc:492] T 9ffc36515643440d8e316a877d6586d7 P ae4c5de7483641ffb0fd3735fd550767: No bootstrap required, opened a new log
I20260811 12:03:34.143004 23281 ts_tablet_manager.cc:1397] T 9ffc36515643440d8e316a877d6586d7 P ae4c5de7483641ffb0fd3735fd550767: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.004s
I20260811 12:03:34.143824 23281 raft_consensus.cc:348] T 9ffc36515643440d8e316a877d6586d7 P ae4c5de7483641ffb0fd3735fd550767 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } }
I20260811 12:03:34.144083 23281 raft_consensus.cc:374] T 9ffc36515643440d8e316a877d6586d7 P ae4c5de7483641ffb0fd3735fd550767 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:03:34.144164 23281 raft_consensus.cc:729] T 9ffc36515643440d8e316a877d6586d7 P ae4c5de7483641ffb0fd3735fd550767 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae4c5de7483641ffb0fd3735fd550767, State: Initialized, Role: FOLLOWER
I20260811 12:03:34.144404 23281 consensus_queue.cc:260] T 9ffc36515643440d8e316a877d6586d7 P ae4c5de7483641ffb0fd3735fd550767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } }
I20260811 12:03:34.144778 23288 tablet_replica.cc:439] T 9ffc36515643440d8e316a877d6586d7 P ae4c5de7483641ffb0fd3735fd550767: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } } }
I20260811 12:03:34.145025 23281 ts_tablet_manager.cc:1428] T 9ffc36515643440d8e316a877d6586d7 P ae4c5de7483641ffb0fd3735fd550767: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 12:03:34.145087 23288 tablet_replica.cc:442] T 9ffc36515643440d8e316a877d6586d7 P ae4c5de7483641ffb0fd3735fd550767: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:03:34.189134 23286 raft_consensus.cc:482] T 9ffc36515643440d8e316a877d6586d7 P af341a2adada48adaa44eaa31c48cc05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:03:34.189333 23286 raft_consensus.cc:504] T 9ffc36515643440d8e316a877d6586d7 P af341a2adada48adaa44eaa31c48cc05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } }
I20260811 12:03:34.190898 23286 leader_election.cc:288] T 9ffc36515643440d8e316a877d6586d7 P af341a2adada48adaa44eaa31c48cc05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9265c12d3059486a8a74caa87ca611d4 (127.9.221.129:38995), ae4c5de7483641ffb0fd3735fd550767 (127.9.221.131:43229)
I20260811 12:03:34.199091 23340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:34.191118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:38995 (local address 127.9.221.130:35511)
0811 12:03:34.191702 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:34.191722 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:34.191866 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:34.192569 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:34.192576 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:34.192617 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:03:34.193160 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:34.193175 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:34.194757 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:34.194761 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:34.196655 (+ 1894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:34.196667 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:34.198073 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:34.198079 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:34.198368 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:34.198394 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 12:03:34.198535 (+ 141us) client_negotiation.cc:240] Negotiation successful
0811 12:03:34.198662 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"mutex_wait_us":27,"thread_start_us":399,"threads_started":1}
I20260811 12:03:34.199087 23269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:34.191236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:35511 (local address 127.9.221.129:38995)
0811 12:03:34.191501 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:34.191508 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:34.191817 (+ 309us) server_negotiation.cc:407] Connection header received
0811 12:03:34.192067 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:34.192076 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:34.192183 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:34.192273 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:03:34.193393 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:34.194604 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:34.196811 (+ 2207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:34.197835 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:34.198034 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:34.198109 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:34.198563 (+ 454us) server_negotiation.cc:299] Negotiation successful
0811 12:03:34.198755 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 12:03:34.200173 22763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ffc36515643440d8e316a877d6586d7" candidate_uuid: "af341a2adada48adaa44eaa31c48cc05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9265c12d3059486a8a74caa87ca611d4" is_pre_election: true
I20260811 12:03:34.200480 22763 raft_consensus.cc:2454] T 9ffc36515643440d8e316a877d6586d7 P 9265c12d3059486a8a74caa87ca611d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af341a2adada48adaa44eaa31c48cc05 in term 0.
I20260811 12:03:34.201014 22858 leader_election.cc:302] T 9ffc36515643440d8e316a877d6586d7 P af341a2adada48adaa44eaa31c48cc05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9265c12d3059486a8a74caa87ca611d4, af341a2adada48adaa44eaa31c48cc05; no voters:
I20260811 12:03:34.201350 23276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:34.192106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:34447 (local address 127.9.221.131:43229)
0811 12:03:34.192292 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:34.192299 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:34.193895 (+ 1596us) server_negotiation.cc:407] Connection header received
0811 12:03:34.194083 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:34.194087 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:34.194145 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:34.194239 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:03:34.194730 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:34.195892 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:34.198634 (+ 2742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:34.200065 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:34.200218 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:34.200290 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:34.200911 (+ 621us) server_negotiation.cc:299] Negotiation successful
0811 12:03:34.201061 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 12:03:34.201741 23286 raft_consensus.cc:2781] T 9ffc36515643440d8e316a877d6586d7 P af341a2adada48adaa44eaa31c48cc05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:03:34.201949 23286 raft_consensus.cc:482] T 9ffc36515643440d8e316a877d6586d7 P af341a2adada48adaa44eaa31c48cc05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:03:34.202122 23286 raft_consensus.cc:3037] T 9ffc36515643440d8e316a877d6586d7 P af341a2adada48adaa44eaa31c48cc05 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:03:34.202309 23342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:34.191888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.131:43229 (local address 127.9.221.130:34447)
0811 12:03:34.193791 (+ 1903us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:34.193804 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:34.193940 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:34.194252 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:34.194257 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:34.194279 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:03:34.194580 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:34.194588 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:34.196053 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:34.196059 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:34.198431 (+ 2372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:34.198449 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:34.200519 (+ 2070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:34.200525 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:34.200699 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:34.200744 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 12:03:34.201612 (+ 868us) client_negotiation.cc:240] Negotiation successful
0811 12:03:34.201752 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1801,"mutex_wait_us":53,"thread_start_us":1612,"threads_started":1}
I20260811 12:03:34.202943 23172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ffc36515643440d8e316a877d6586d7" candidate_uuid: "af341a2adada48adaa44eaa31c48cc05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae4c5de7483641ffb0fd3735fd550767" is_pre_election: true
I20260811 12:03:34.203172 23172 raft_consensus.cc:2454] T 9ffc36515643440d8e316a877d6586d7 P ae4c5de7483641ffb0fd3735fd550767 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af341a2adada48adaa44eaa31c48cc05 in term 0.
I20260811 12:03:34.204862 23286 raft_consensus.cc:504] T 9ffc36515643440d8e316a877d6586d7 P af341a2adada48adaa44eaa31c48cc05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } }
I20260811 12:03:34.205406 23286 leader_election.cc:288] T 9ffc36515643440d8e316a877d6586d7 P af341a2adada48adaa44eaa31c48cc05 [CANDIDATE]: Term 1 election: Requested vote from peers 9265c12d3059486a8a74caa87ca611d4 (127.9.221.129:38995), ae4c5de7483641ffb0fd3735fd550767 (127.9.221.131:43229)
I20260811 12:03:34.205847 22763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ffc36515643440d8e316a877d6586d7" candidate_uuid: "af341a2adada48adaa44eaa31c48cc05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9265c12d3059486a8a74caa87ca611d4"
I20260811 12:03:34.206117 22763 raft_consensus.cc:3037] T 9ffc36515643440d8e316a877d6586d7 P 9265c12d3059486a8a74caa87ca611d4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:03:34.208351 22763 raft_consensus.cc:2454] T 9ffc36515643440d8e316a877d6586d7 P 9265c12d3059486a8a74caa87ca611d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af341a2adada48adaa44eaa31c48cc05 in term 1.
I20260811 12:03:34.208762 22858 leader_election.cc:302] T 9ffc36515643440d8e316a877d6586d7 P af341a2adada48adaa44eaa31c48cc05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9265c12d3059486a8a74caa87ca611d4, af341a2adada48adaa44eaa31c48cc05; no voters:
I20260811 12:03:34.208758 23172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ffc36515643440d8e316a877d6586d7" candidate_uuid: "af341a2adada48adaa44eaa31c48cc05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae4c5de7483641ffb0fd3735fd550767"
I20260811 12:03:34.208976 23172 raft_consensus.cc:3037] T 9ffc36515643440d8e316a877d6586d7 P ae4c5de7483641ffb0fd3735fd550767 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:03:34.209049 23286 raft_consensus.cc:2781] T 9ffc36515643440d8e316a877d6586d7 P af341a2adada48adaa44eaa31c48cc05 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:03:34.209553 23286 raft_consensus.cc:686] T 9ffc36515643440d8e316a877d6586d7 P af341a2adada48adaa44eaa31c48cc05 [term 1 LEADER]: Becoming Leader. State: Replica: af341a2adada48adaa44eaa31c48cc05, State: Running, Role: LEADER
I20260811 12:03:34.209967 23286 consensus_queue.cc:237] T 9ffc36515643440d8e316a877d6586d7 P af341a2adada48adaa44eaa31c48cc05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } }
I20260811 12:03:34.212037 23172 raft_consensus.cc:2454] T 9ffc36515643440d8e316a877d6586d7 P ae4c5de7483641ffb0fd3735fd550767 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af341a2adada48adaa44eaa31c48cc05 in term 1.
I20260811 12:03:34.213267 23343 tablet_replica.cc:439] T 9ffc36515643440d8e316a877d6586d7 P af341a2adada48adaa44eaa31c48cc05: TxnStatusTablet state changed. Reason: New leader af341a2adada48adaa44eaa31c48cc05. Latest consensus state: current_term: 1 leader_uuid: "af341a2adada48adaa44eaa31c48cc05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } } }
I20260811 12:03:34.213898 23343 tablet_replica.cc:442] T 9ffc36515643440d8e316a877d6586d7 P af341a2adada48adaa44eaa31c48cc05: This TxnStatusTablet replica's current role is: LEADER
I20260811 12:03:34.214686 23345 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 12:03:34.214907 23345 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 12:03:34.214864 22531 catalog_manager.cc:5495] T 9ffc36515643440d8e316a877d6586d7 P af341a2adada48adaa44eaa31c48cc05 reported cstate change: term changed from 0 to 1, leader changed from <none> to af341a2adada48adaa44eaa31c48cc05 (127.9.221.130). New cstate: current_term: 1 leader_uuid: "af341a2adada48adaa44eaa31c48cc05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } health_report { overall_health: UNKNOWN } } }
I20260811 12:03:34.233377 23332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:34.227418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.130:46333 (local address 127.0.0.1:59726)
0811 12:03:34.227793 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:34.227809 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:34.227978 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:34.228433 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:34.228438 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:34.228451 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 12:03:34.228641 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:34.228648 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:34.229682 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:34.229689 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:34.230896 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:34.230908 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:34.231790 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:34.231796 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:34.231970 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:34.232022 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 12:03:34.232813 (+ 791us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 12:03:34.232834 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 12:03:34.232915 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 12:03:34.233086 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191}
I20260811 12:03:34.233371 23273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:34.227438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59726 (local address 127.9.221.130:46333)
0811 12:03:34.227629 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:34.227635 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:34.227881 (+ 246us) server_negotiation.cc:407] Connection header received
0811 12:03:34.228171 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:34.228176 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:34.228277 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:34.228415 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:03:34.228789 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:34.229539 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:34.231047 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:34.231603 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:34.231767 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:34.232226 (+ 459us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:03:34.232694 (+ 468us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:03:34.232839 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:34.232935 (+ 96us) server_negotiation.cc:299] Negotiation successful
0811 12:03:34.233015 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 12:03:34.237457 23172 raft_consensus.cc:1261] T 9ffc36515643440d8e316a877d6586d7 P ae4c5de7483641ffb0fd3735fd550767 [term 1 FOLLOWER]: Refusing update from remote peer af341a2adada48adaa44eaa31c48cc05: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:03:34.237833 22763 raft_consensus.cc:1261] T 9ffc36515643440d8e316a877d6586d7 P 9265c12d3059486a8a74caa87ca611d4 [term 1 FOLLOWER]: Refusing update from remote peer af341a2adada48adaa44eaa31c48cc05: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:03:34.238850 23343 consensus_queue.cc:1035] T 9ffc36515643440d8e316a877d6586d7 P af341a2adada48adaa44eaa31c48cc05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:03:34.239132 23286 consensus_queue.cc:1035] T 9ffc36515643440d8e316a877d6586d7 P af341a2adada48adaa44eaa31c48cc05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:03:34.242952 23288 tablet_replica.cc:439] T 9ffc36515643440d8e316a877d6586d7 P ae4c5de7483641ffb0fd3735fd550767: TxnStatusTablet state changed. Reason: New leader af341a2adada48adaa44eaa31c48cc05. Latest consensus state: current_term: 1 leader_uuid: "af341a2adada48adaa44eaa31c48cc05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } } }
I20260811 12:03:34.243311 23288 tablet_replica.cc:442] T 9ffc36515643440d8e316a877d6586d7 P ae4c5de7483641ffb0fd3735fd550767: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:03:34.247076 23296 tablet_replica.cc:439] T 9ffc36515643440d8e316a877d6586d7 P 9265c12d3059486a8a74caa87ca611d4: TxnStatusTablet state changed. Reason: New leader af341a2adada48adaa44eaa31c48cc05. Latest consensus state: current_term: 1 leader_uuid: "af341a2adada48adaa44eaa31c48cc05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } } }
I20260811 12:03:34.247372 23296 tablet_replica.cc:442] T 9ffc36515643440d8e316a877d6586d7 P 9265c12d3059486a8a74caa87ca611d4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:03:34.249425 23343 tablet_replica.cc:439] T 9ffc36515643440d8e316a877d6586d7 P af341a2adada48adaa44eaa31c48cc05: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "af341a2adada48adaa44eaa31c48cc05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } } }
I20260811 12:03:34.249716 23343 tablet_replica.cc:442] T 9ffc36515643440d8e316a877d6586d7 P af341a2adada48adaa44eaa31c48cc05: This TxnStatusTablet replica's current role is: LEADER
I20260811 12:03:34.249953 23306 mvcc.cc:205] Tried to move back new op lower bound from 7317298439109525504 to 7317298439008006144. Current Snapshot: MvccSnapshot[applied={T|T < 7317298439109525504 or (T in {7317298439109525504})}]
I20260811 12:03:34.251984 23286 tablet_replica.cc:439] T 9ffc36515643440d8e316a877d6586d7 P af341a2adada48adaa44eaa31c48cc05: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "af341a2adada48adaa44eaa31c48cc05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } } }
I20260811 12:03:34.252276 23286 tablet_replica.cc:442] T 9ffc36515643440d8e316a877d6586d7 P af341a2adada48adaa44eaa31c48cc05: This TxnStatusTablet replica's current role is: LEADER
I20260811 12:03:34.253538 23288 tablet_replica.cc:439] T 9ffc36515643440d8e316a877d6586d7 P ae4c5de7483641ffb0fd3735fd550767: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af341a2adada48adaa44eaa31c48cc05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } } }
I20260811 12:03:34.254740 23288 tablet_replica.cc:442] T 9ffc36515643440d8e316a877d6586d7 P ae4c5de7483641ffb0fd3735fd550767: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:03:34.258021 23296 tablet_replica.cc:439] T 9ffc36515643440d8e316a877d6586d7 P 9265c12d3059486a8a74caa87ca611d4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af341a2adada48adaa44eaa31c48cc05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } } }
I20260811 12:03:34.258309 23296 tablet_replica.cc:442] T 9ffc36515643440d8e316a877d6586d7 P 9265c12d3059486a8a74caa87ca611d4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:03:34.265591 23276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:34.258362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55464 (local address 127.9.221.131:43229)
0811 12:03:34.258526 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:34.258532 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:34.258696 (+ 164us) server_negotiation.cc:407] Connection header received
0811 12:03:34.258932 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:34.258937 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:34.259064 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:34.259226 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:03:34.259738 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:34.260968 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:34.262999 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:34.263712 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:34.263858 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:34.264286 (+ 428us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:03:34.264995 (+ 709us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:03:34.265195 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:34.265246 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 12:03:34.265325 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 12:03:34.281152 23273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:34.275392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:54099 (local address 127.9.221.130:46333)
0811 12:03:34.275623 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:34.275631 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:34.276252 (+ 621us) server_negotiation.cc:407] Connection header received
0811 12:03:34.276470 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:34.276476 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:34.276565 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:34.276652 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:03:34.277167 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:34.277921 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:34.279255 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:34.279836 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:34.279967 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:34.280385 (+ 418us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:03:34.280488 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:03:34.280589 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:34.280733 (+ 144us) server_negotiation.cc:299] Negotiation successful
0811 12:03:34.280814 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 12:03:34.281234 23357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:34.275297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.130:46333 (local address 127.9.221.131:54099)
0811 12:03:34.276149 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:34.276165 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:34.276319 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:34.276680 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:34.276684 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:34.276696 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 12:03:34.277003 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:34.277011 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:34.278048 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:34.278055 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:34.279117 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:34.279128 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:34.280017 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:34.280025 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:34.280159 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:34.280210 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 12:03:34.280588 (+ 378us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 12:03:34.280601 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 12:03:34.280706 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 12:03:34.280836 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":677,"thread_start_us":534,"threads_started":1}
I20260811 12:03:34.292358 22960 raft_consensus.cc:1261] T 46f67bffd7364bbc89237072d52ec9fa P af341a2adada48adaa44eaa31c48cc05 [term 1 FOLLOWER]: Refusing update from remote peer ae4c5de7483641ffb0fd3735fd550767: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:03:34.292510 22763 raft_consensus.cc:1261] T 46f67bffd7364bbc89237072d52ec9fa P 9265c12d3059486a8a74caa87ca611d4 [term 1 FOLLOWER]: Refusing update from remote peer ae4c5de7483641ffb0fd3735fd550767: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:03:34.293125 23323 consensus_queue.cc:1035] T 46f67bffd7364bbc89237072d52ec9fa P ae4c5de7483641ffb0fd3735fd550767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:03:34.293852 23288 consensus_queue.cc:1035] T 46f67bffd7364bbc89237072d52ec9fa P ae4c5de7483641ffb0fd3735fd550767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:03:34.360500 23276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:34.343707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:45643 (local address 127.9.221.131:43229)
0811 12:03:34.344026 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:34.344035 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:34.352348 (+ 8313us) server_negotiation.cc:407] Connection header received
0811 12:03:34.352558 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:34.352563 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:34.352636 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:34.352733 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:03:34.353554 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:34.354496 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:34.358144 (+ 3648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:34.358864 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:34.358964 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:34.359614 (+ 650us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:03:34.359774 (+ 160us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:03:34.359842 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:34.360090 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 12:03:34.360174 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164}
I20260811 12:03:34.362001 23366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:34.343921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.131:43229 (local address 127.9.221.130:45643)
0811 12:03:34.352238 (+ 8317us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:34.352255 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:34.352422 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:34.352748 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:34.352753 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:34.352764 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 12:03:34.353196 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:34.353210 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:34.356144 (+ 2934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:34.356149 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:34.357983 (+ 1834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:34.357999 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:34.359162 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:34.359168 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:34.359338 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:34.359404 (+ 66us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 12:03:34.359961 (+ 557us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 12:03:34.359971 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 12:03:34.361014 (+ 1043us) client_negotiation.cc:240] Negotiation successful
0811 12:03:34.361168 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8136,"thread_start_us":7971,"threads_started":1}
I20260811 12:03:34.414361 22531 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:03:34.414630 22531 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:03:34.418164 22531 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bac306678bd5437a8b1101dae6b4b615: Sending DeleteTablet for 3 replicas of tablet 46f67bffd7364bbc89237072d52ec9fa
I20260811 12:03:34.419163 22929 tablet_service.cc:1514] Processing DeleteTablet for tablet 46f67bffd7364bbc89237072d52ec9fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:03:34 UTC) from {username='oryx', principal='oryx/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:59724
I20260811 12:03:34.419288 22726 tablet_service.cc:1514] Processing DeleteTablet for tablet 46f67bffd7364bbc89237072d52ec9fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:03:34 UTC) from {username='oryx', principal='oryx/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:33434
I20260811 12:03:34.419701 23142 tablet_service.cc:1514] Processing DeleteTablet for tablet 46f67bffd7364bbc89237072d52ec9fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:03:34 UTC) from {username='oryx', principal='oryx/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:55462
I20260811 12:03:34.436677 23374 tablet_replica.cc:330] T 46f67bffd7364bbc89237072d52ec9fa P ae4c5de7483641ffb0fd3735fd550767: stopping tablet replica
I20260811 12:03:34.436971 23374 raft_consensus.cc:2229] T 46f67bffd7364bbc89237072d52ec9fa P ae4c5de7483641ffb0fd3735fd550767 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:03:34.444160 23374 raft_consensus.cc:2258] T 46f67bffd7364bbc89237072d52ec9fa P ae4c5de7483641ffb0fd3735fd550767 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:03:34.447213 23373 tablet_replica.cc:330] T 46f67bffd7364bbc89237072d52ec9fa P 9265c12d3059486a8a74caa87ca611d4: stopping tablet replica
I20260811 12:03:34.447443 23373 raft_consensus.cc:2229] T 46f67bffd7364bbc89237072d52ec9fa P 9265c12d3059486a8a74caa87ca611d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:03:34.447734 23373 raft_consensus.cc:2258] T 46f67bffd7364bbc89237072d52ec9fa P 9265c12d3059486a8a74caa87ca611d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:03:34.449491 23374 ts_tablet_manager.cc:1905] T 46f67bffd7364bbc89237072d52ec9fa P ae4c5de7483641ffb0fd3735fd550767: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:03:34.449682 23373 ts_tablet_manager.cc:1905] T 46f67bffd7364bbc89237072d52ec9fa P 9265c12d3059486a8a74caa87ca611d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:03:34.456851 23372 tablet_replica.cc:330] T 46f67bffd7364bbc89237072d52ec9fa P af341a2adada48adaa44eaa31c48cc05: stopping tablet replica
I20260811 12:03:34.457053 23372 raft_consensus.cc:2229] T 46f67bffd7364bbc89237072d52ec9fa P af341a2adada48adaa44eaa31c48cc05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:03:34.457317 23372 raft_consensus.cc:2258] T 46f67bffd7364bbc89237072d52ec9fa P af341a2adada48adaa44eaa31c48cc05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:03:34.457634 23374 ts_tablet_manager.cc:1918] T 46f67bffd7364bbc89237072d52ec9fa P ae4c5de7483641ffb0fd3735fd550767: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:03:34.457758 23374 log.cc:1196] T 46f67bffd7364bbc89237072d52ec9fa P ae4c5de7483641ffb0fd3735fd550767: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-2/wal/wals/46f67bffd7364bbc89237072d52ec9fa
I20260811 12:03:34.458238 23374 ts_tablet_manager.cc:1939] T 46f67bffd7364bbc89237072d52ec9fa P ae4c5de7483641ffb0fd3735fd550767: Deleting consensus metadata
I20260811 12:03:34.459393 22515 catalog_manager.cc:4841] TS ae4c5de7483641ffb0fd3735fd550767 (127.9.221.131:43229): tablet 46f67bffd7364bbc89237072d52ec9fa (table test-table [id=a1296519484b48e49830f76aab9f522e]) successfully deleted
I20260811 12:03:34.465765 23372 ts_tablet_manager.cc:1905] T 46f67bffd7364bbc89237072d52ec9fa P af341a2adada48adaa44eaa31c48cc05: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:03:34.475481 23373 ts_tablet_manager.cc:1918] T 46f67bffd7364bbc89237072d52ec9fa P 9265c12d3059486a8a74caa87ca611d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:03:34.475658 23373 log.cc:1196] T 46f67bffd7364bbc89237072d52ec9fa P 9265c12d3059486a8a74caa87ca611d4: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-0/wal/wals/46f67bffd7364bbc89237072d52ec9fa
I20260811 12:03:34.476158 23373 ts_tablet_manager.cc:1939] T 46f67bffd7364bbc89237072d52ec9fa P 9265c12d3059486a8a74caa87ca611d4: Deleting consensus metadata
I20260811 12:03:34.477396 22515 catalog_manager.cc:4841] TS 9265c12d3059486a8a74caa87ca611d4 (127.9.221.129:38995): tablet 46f67bffd7364bbc89237072d52ec9fa (table test-table [id=a1296519484b48e49830f76aab9f522e]) successfully deleted
Aug 11 12:03:34 dist-test-slave-mjjr krb5kdc[22459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449814, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
I20260811 12:03:34.485491 23372 ts_tablet_manager.cc:1918] T 46f67bffd7364bbc89237072d52ec9fa P af341a2adada48adaa44eaa31c48cc05: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:03:34.485646 23372 log.cc:1196] T 46f67bffd7364bbc89237072d52ec9fa P af341a2adada48adaa44eaa31c48cc05: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-1/wal/wals/46f67bffd7364bbc89237072d52ec9fa
I20260811 12:03:34.486132 23372 ts_tablet_manager.cc:1939] T 46f67bffd7364bbc89237072d52ec9fa P af341a2adada48adaa44eaa31c48cc05: Deleting consensus metadata
I20260811 12:03:34.487368 22513 catalog_manager.cc:4841] TS af341a2adada48adaa44eaa31c48cc05 (127.9.221.130:46333): tablet 46f67bffd7364bbc89237072d52ec9fa (table test-table [id=a1296519484b48e49830f76aab9f522e]) successfully deleted
Aug 11 12:03:34 dist-test-slave-mjjr krb5kdc[22459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:34 dist-test-slave-mjjr krb5kdc[22459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 12:03:34.535427 22665 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:34.521391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33254 (local address 127.9.221.190:35905)
0811 12:03:34.521594 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:34.521601 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:34.522200 (+ 599us) server_negotiation.cc:407] Connection header received
0811 12:03:34.522391 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:34.522398 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:34.522491 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:34.522612 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:34.524416 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:34.525623 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:34.527514 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:34.528244 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:34.528380 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:34.535150 (+ 6770us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33254: BlockingRecv error: recv got EOF from 127.0.0.1:33254 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 11 12:03:34 dist-test-slave-mjjr krb5kdc[22459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449814, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM
I20260811 12:03:34.556857 22665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:34.539356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33256 (local address 127.9.221.190:35905)
0811 12:03:34.539542 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:34.539548 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:34.540136 (+ 588us) server_negotiation.cc:407] Connection header received
0811 12:03:34.540305 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:34.540310 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:34.540395 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:34.540497 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:34.542051 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:34.543117 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:34.544845 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:34.545527 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:34.545664 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:34.550399 (+ 4735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:34.550406 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:34.550413 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:34.550473 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:34.553881 (+ 3408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:34.554694 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:34.554700 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:34.554703 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:34.554857 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:34.555274 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:34.555280 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:34.555285 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:34.555628 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:34.555910 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:34.556221 (+ 311us) server_negotiation.cc:299] Negotiation successful
0811 12:03:34.556450 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 12:03:34.560609 22531 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33256:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:03:34.561373 22531 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:03:34.570313 22929 tablet_service.cc:1467] Processing CreateTablet for tablet 4c34798f99d24a489a05958d610fbffb (DEFAULT_TABLE table=test-table [id=5a53782293dc4ce9a2cbe8753f3d2634]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:03:34.570343 22726 tablet_service.cc:1467] Processing CreateTablet for tablet 4c34798f99d24a489a05958d610fbffb (DEFAULT_TABLE table=test-table [id=5a53782293dc4ce9a2cbe8753f3d2634]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:03:34.570822 22929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c34798f99d24a489a05958d610fbffb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:03:34.570847 22726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c34798f99d24a489a05958d610fbffb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:03:34.576143 23142 tablet_service.cc:1467] Processing CreateTablet for tablet 4c34798f99d24a489a05958d610fbffb (DEFAULT_TABLE table=test-table [id=5a53782293dc4ce9a2cbe8753f3d2634]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:03:34.576694 23142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c34798f99d24a489a05958d610fbffb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:03:34.579960 23280 tablet_bootstrap.cc:492] T 4c34798f99d24a489a05958d610fbffb P af341a2adada48adaa44eaa31c48cc05: Bootstrap starting.
I20260811 12:03:34.581703 23280 tablet_bootstrap.cc:654] T 4c34798f99d24a489a05958d610fbffb P af341a2adada48adaa44eaa31c48cc05: Neither blocks nor log segments found. Creating new log.
I20260811 12:03:34.585219 23281 tablet_bootstrap.cc:492] T 4c34798f99d24a489a05958d610fbffb P ae4c5de7483641ffb0fd3735fd550767: Bootstrap starting.
I20260811 12:03:34.586858 23283 tablet_bootstrap.cc:492] T 4c34798f99d24a489a05958d610fbffb P 9265c12d3059486a8a74caa87ca611d4: Bootstrap starting.
I20260811 12:03:34.586936 23281 tablet_bootstrap.cc:654] T 4c34798f99d24a489a05958d610fbffb P ae4c5de7483641ffb0fd3735fd550767: Neither blocks nor log segments found. Creating new log.
I20260811 12:03:34.588681 23283 tablet_bootstrap.cc:654] T 4c34798f99d24a489a05958d610fbffb P 9265c12d3059486a8a74caa87ca611d4: Neither blocks nor log segments found. Creating new log.
I20260811 12:03:34.590145 23280 tablet_bootstrap.cc:492] T 4c34798f99d24a489a05958d610fbffb P af341a2adada48adaa44eaa31c48cc05: No bootstrap required, opened a new log
I20260811 12:03:34.590340 23280 ts_tablet_manager.cc:1397] T 4c34798f99d24a489a05958d610fbffb P af341a2adada48adaa44eaa31c48cc05: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.001s
I20260811 12:03:34.591236 23280 raft_consensus.cc:348] T 4c34798f99d24a489a05958d610fbffb P af341a2adada48adaa44eaa31c48cc05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } }
I20260811 12:03:34.591579 23280 raft_consensus.cc:374] T 4c34798f99d24a489a05958d610fbffb P af341a2adada48adaa44eaa31c48cc05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:03:34.591733 23280 raft_consensus.cc:729] T 4c34798f99d24a489a05958d610fbffb P af341a2adada48adaa44eaa31c48cc05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af341a2adada48adaa44eaa31c48cc05, State: Initialized, Role: FOLLOWER
I20260811 12:03:34.592199 23280 consensus_queue.cc:260] T 4c34798f99d24a489a05958d610fbffb P af341a2adada48adaa44eaa31c48cc05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } }
I20260811 12:03:34.594939 23280 ts_tablet_manager.cc:1428] T 4c34798f99d24a489a05958d610fbffb P af341a2adada48adaa44eaa31c48cc05: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 12:03:34.604647 23283 tablet_bootstrap.cc:492] T 4c34798f99d24a489a05958d610fbffb P 9265c12d3059486a8a74caa87ca611d4: No bootstrap required, opened a new log
I20260811 12:03:34.604833 23283 ts_tablet_manager.cc:1397] T 4c34798f99d24a489a05958d610fbffb P 9265c12d3059486a8a74caa87ca611d4: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.005s
I20260811 12:03:34.605722 23283 raft_consensus.cc:348] T 4c34798f99d24a489a05958d610fbffb P 9265c12d3059486a8a74caa87ca611d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } }
I20260811 12:03:34.606027 23283 raft_consensus.cc:374] T 4c34798f99d24a489a05958d610fbffb P 9265c12d3059486a8a74caa87ca611d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:03:34.606149 23283 raft_consensus.cc:729] T 4c34798f99d24a489a05958d610fbffb P 9265c12d3059486a8a74caa87ca611d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9265c12d3059486a8a74caa87ca611d4, State: Initialized, Role: FOLLOWER
I20260811 12:03:34.606452 23283 consensus_queue.cc:260] T 4c34798f99d24a489a05958d610fbffb P 9265c12d3059486a8a74caa87ca611d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } }
I20260811 12:03:34.608239 23283 ts_tablet_manager.cc:1428] T 4c34798f99d24a489a05958d610fbffb P 9265c12d3059486a8a74caa87ca611d4: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 12:03:34.611720 23281 tablet_bootstrap.cc:492] T 4c34798f99d24a489a05958d610fbffb P ae4c5de7483641ffb0fd3735fd550767: No bootstrap required, opened a new log
I20260811 12:03:34.611881 23281 ts_tablet_manager.cc:1397] T 4c34798f99d24a489a05958d610fbffb P ae4c5de7483641ffb0fd3735fd550767: Time spent bootstrapping tablet: real 0.027s user 0.005s sys 0.011s
I20260811 12:03:34.612689 23281 raft_consensus.cc:348] T 4c34798f99d24a489a05958d610fbffb P ae4c5de7483641ffb0fd3735fd550767 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } }
I20260811 12:03:34.612977 23281 raft_consensus.cc:374] T 4c34798f99d24a489a05958d610fbffb P ae4c5de7483641ffb0fd3735fd550767 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:03:34.613076 23281 raft_consensus.cc:729] T 4c34798f99d24a489a05958d610fbffb P ae4c5de7483641ffb0fd3735fd550767 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae4c5de7483641ffb0fd3735fd550767, State: Initialized, Role: FOLLOWER
I20260811 12:03:34.613328 23281 consensus_queue.cc:260] T 4c34798f99d24a489a05958d610fbffb P ae4c5de7483641ffb0fd3735fd550767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } }
I20260811 12:03:34.614010 23281 ts_tablet_manager.cc:1428] T 4c34798f99d24a489a05958d610fbffb P ae4c5de7483641ffb0fd3735fd550767: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 12:03:34.661396 23288 raft_consensus.cc:482] T 4c34798f99d24a489a05958d610fbffb P ae4c5de7483641ffb0fd3735fd550767 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:03:34.661674 23288 raft_consensus.cc:504] T 4c34798f99d24a489a05958d610fbffb P ae4c5de7483641ffb0fd3735fd550767 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } }
I20260811 12:03:34.662562 23288 leader_election.cc:288] T 4c34798f99d24a489a05958d610fbffb P ae4c5de7483641ffb0fd3735fd550767 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9265c12d3059486a8a74caa87ca611d4 (127.9.221.129:38995), af341a2adada48adaa44eaa31c48cc05 (127.9.221.130:46333)
I20260811 12:03:34.663009 22763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c34798f99d24a489a05958d610fbffb" candidate_uuid: "ae4c5de7483641ffb0fd3735fd550767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9265c12d3059486a8a74caa87ca611d4" is_pre_election: true
I20260811 12:03:34.663247 22763 raft_consensus.cc:2454] T 4c34798f99d24a489a05958d610fbffb P 9265c12d3059486a8a74caa87ca611d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae4c5de7483641ffb0fd3735fd550767 in term 0.
I20260811 12:03:34.663389 22960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c34798f99d24a489a05958d610fbffb" candidate_uuid: "ae4c5de7483641ffb0fd3735fd550767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af341a2adada48adaa44eaa31c48cc05" is_pre_election: true
I20260811 12:03:34.663724 22960 raft_consensus.cc:2454] T 4c34798f99d24a489a05958d610fbffb P af341a2adada48adaa44eaa31c48cc05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae4c5de7483641ffb0fd3735fd550767 in term 0.
I20260811 12:03:34.663933 23064 leader_election.cc:302] T 4c34798f99d24a489a05958d610fbffb P ae4c5de7483641ffb0fd3735fd550767 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9265c12d3059486a8a74caa87ca611d4, ae4c5de7483641ffb0fd3735fd550767; no voters:
I20260811 12:03:34.664307 23288 raft_consensus.cc:2781] T 4c34798f99d24a489a05958d610fbffb P ae4c5de7483641ffb0fd3735fd550767 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:03:34.664448 23288 raft_consensus.cc:482] T 4c34798f99d24a489a05958d610fbffb P ae4c5de7483641ffb0fd3735fd550767 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:03:34.664543 23288 raft_consensus.cc:3037] T 4c34798f99d24a489a05958d610fbffb P ae4c5de7483641ffb0fd3735fd550767 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:03:34.667706 23288 raft_consensus.cc:504] T 4c34798f99d24a489a05958d610fbffb P ae4c5de7483641ffb0fd3735fd550767 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } }
I20260811 12:03:34.668720 23288 leader_election.cc:288] T 4c34798f99d24a489a05958d610fbffb P ae4c5de7483641ffb0fd3735fd550767 [CANDIDATE]: Term 1 election: Requested vote from peers 9265c12d3059486a8a74caa87ca611d4 (127.9.221.129:38995), af341a2adada48adaa44eaa31c48cc05 (127.9.221.130:46333)
I20260811 12:03:34.668869 22763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c34798f99d24a489a05958d610fbffb" candidate_uuid: "ae4c5de7483641ffb0fd3735fd550767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9265c12d3059486a8a74caa87ca611d4"
I20260811 12:03:34.669087 22763 raft_consensus.cc:3037] T 4c34798f99d24a489a05958d610fbffb P 9265c12d3059486a8a74caa87ca611d4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:03:34.671538 22960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c34798f99d24a489a05958d610fbffb" candidate_uuid: "ae4c5de7483641ffb0fd3735fd550767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af341a2adada48adaa44eaa31c48cc05"
I20260811 12:03:34.671779 22960 raft_consensus.cc:3037] T 4c34798f99d24a489a05958d610fbffb P af341a2adada48adaa44eaa31c48cc05 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:03:34.672355 22763 raft_consensus.cc:2454] T 4c34798f99d24a489a05958d610fbffb P 9265c12d3059486a8a74caa87ca611d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae4c5de7483641ffb0fd3735fd550767 in term 1.
I20260811 12:03:34.672827 23064 leader_election.cc:302] T 4c34798f99d24a489a05958d610fbffb P ae4c5de7483641ffb0fd3735fd550767 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9265c12d3059486a8a74caa87ca611d4, ae4c5de7483641ffb0fd3735fd550767; no voters:
I20260811 12:03:34.673250 23288 raft_consensus.cc:2781] T 4c34798f99d24a489a05958d610fbffb P ae4c5de7483641ffb0fd3735fd550767 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:03:34.674896 22960 raft_consensus.cc:2454] T 4c34798f99d24a489a05958d610fbffb P af341a2adada48adaa44eaa31c48cc05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae4c5de7483641ffb0fd3735fd550767 in term 1.
I20260811 12:03:34.675988 23288 raft_consensus.cc:686] T 4c34798f99d24a489a05958d610fbffb P ae4c5de7483641ffb0fd3735fd550767 [term 1 LEADER]: Becoming Leader. State: Replica: ae4c5de7483641ffb0fd3735fd550767, State: Running, Role: LEADER
I20260811 12:03:34.676494 23288 consensus_queue.cc:237] T 4c34798f99d24a489a05958d610fbffb P ae4c5de7483641ffb0fd3735fd550767 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } }
I20260811 12:03:34.685518 22531 catalog_manager.cc:5495] T 4c34798f99d24a489a05958d610fbffb P ae4c5de7483641ffb0fd3735fd550767 reported cstate change: term changed from 0 to 1, leader changed from <none> to ae4c5de7483641ffb0fd3735fd550767 (127.9.221.131). New cstate: current_term: 1 leader_uuid: "ae4c5de7483641ffb0fd3735fd550767" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } health_report { overall_health: HEALTHY } } }
I20260811 12:03:34.717588 23276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:34.708988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55472 (local address 127.9.221.131:43229)
0811 12:03:34.709199 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:34.709206 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:34.710410 (+ 1204us) server_negotiation.cc:407] Connection header received
0811 12:03:34.710506 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:34.710511 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:34.710608 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:34.710714 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:03:34.711389 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:34.712603 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:34.714719 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:34.715425 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:34.715559 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:34.716350 (+ 791us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:03:34.716533 (+ 183us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:03:34.716635 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:34.717114 (+ 479us) server_negotiation.cc:299] Negotiation successful
0811 12:03:34.717230 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 12:03:34.721127 22960 raft_consensus.cc:1261] T 4c34798f99d24a489a05958d610fbffb P af341a2adada48adaa44eaa31c48cc05 [term 1 FOLLOWER]: Refusing update from remote peer ae4c5de7483641ffb0fd3735fd550767: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:03:34.721335 22763 raft_consensus.cc:1261] T 4c34798f99d24a489a05958d610fbffb P 9265c12d3059486a8a74caa87ca611d4 [term 1 FOLLOWER]: Refusing update from remote peer ae4c5de7483641ffb0fd3735fd550767: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:03:34.721987 23288 consensus_queue.cc:1035] T 4c34798f99d24a489a05958d610fbffb P ae4c5de7483641ffb0fd3735fd550767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:03:34.722363 23364 consensus_queue.cc:1035] T 4c34798f99d24a489a05958d610fbffb P ae4c5de7483641ffb0fd3735fd550767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:03:34.749104 22535 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:03:34.749430 22535 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:03:34.753381 22535 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bac306678bd5437a8b1101dae6b4b615: Sending DeleteTablet for 3 replicas of tablet 4c34798f99d24a489a05958d610fbffb
I20260811 12:03:34.754379 22726 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c34798f99d24a489a05958d610fbffb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:03:34 UTC) from {username='oryx', principal='oryx/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:33434
I20260811 12:03:34.754683 22929 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c34798f99d24a489a05958d610fbffb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:03:34 UTC) from {username='oryx', principal='oryx/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:59724
I20260811 12:03:34.754959 23373 tablet_replica.cc:330] T 4c34798f99d24a489a05958d610fbffb P 9265c12d3059486a8a74caa87ca611d4: stopping tablet replica
I20260811 12:03:34.754969 23142 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c34798f99d24a489a05958d610fbffb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:03:34 UTC) from {username='oryx', principal='oryx/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:55462
I20260811 12:03:34.755151 23373 raft_consensus.cc:2229] T 4c34798f99d24a489a05958d610fbffb P 9265c12d3059486a8a74caa87ca611d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:03:34.755352 23373 raft_consensus.cc:2258] T 4c34798f99d24a489a05958d610fbffb P 9265c12d3059486a8a74caa87ca611d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:03:34.755620 23372 tablet_replica.cc:330] T 4c34798f99d24a489a05958d610fbffb P af341a2adada48adaa44eaa31c48cc05: stopping tablet replica
I20260811 12:03:34.755837 23372 raft_consensus.cc:2229] T 4c34798f99d24a489a05958d610fbffb P af341a2adada48adaa44eaa31c48cc05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:03:34.756034 23374 tablet_replica.cc:330] T 4c34798f99d24a489a05958d610fbffb P ae4c5de7483641ffb0fd3735fd550767: stopping tablet replica
I20260811 12:03:34.756062 23372 raft_consensus.cc:2258] T 4c34798f99d24a489a05958d610fbffb P af341a2adada48adaa44eaa31c48cc05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:03:34.756299 23374 raft_consensus.cc:2229] T 4c34798f99d24a489a05958d610fbffb P ae4c5de7483641ffb0fd3735fd550767 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:03:34.756740 23374 raft_consensus.cc:2258] T 4c34798f99d24a489a05958d610fbffb P ae4c5de7483641ffb0fd3735fd550767 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:03:34.757287 23373 ts_tablet_manager.cc:1905] T 4c34798f99d24a489a05958d610fbffb P 9265c12d3059486a8a74caa87ca611d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:03:34.758664 23374 ts_tablet_manager.cc:1905] T 4c34798f99d24a489a05958d610fbffb P ae4c5de7483641ffb0fd3735fd550767: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:03:34.761467 22535 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33256:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:03:34.762262 22535 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:03:34.764094 23373 ts_tablet_manager.cc:1918] T 4c34798f99d24a489a05958d610fbffb P 9265c12d3059486a8a74caa87ca611d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:03:34.764245 23373 log.cc:1196] T 4c34798f99d24a489a05958d610fbffb P 9265c12d3059486a8a74caa87ca611d4: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-0/wal/wals/4c34798f99d24a489a05958d610fbffb
I20260811 12:03:34.764729 23373 ts_tablet_manager.cc:1939] T 4c34798f99d24a489a05958d610fbffb P 9265c12d3059486a8a74caa87ca611d4: Deleting consensus metadata
I20260811 12:03:34.765844 22515 catalog_manager.cc:4841] TS 9265c12d3059486a8a74caa87ca611d4 (127.9.221.129:38995): tablet 4c34798f99d24a489a05958d610fbffb (table test-table [id=5a53782293dc4ce9a2cbe8753f3d2634]) successfully deleted
I20260811 12:03:34.766821 23372 ts_tablet_manager.cc:1905] T 4c34798f99d24a489a05958d610fbffb P af341a2adada48adaa44eaa31c48cc05: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:03:34.774235 22726 tablet_service.cc:1467] Processing CreateTablet for tablet 8dc1e34f64f34fb984bc15b7e4a73b67 (DEFAULT_TABLE table=test-table [id=f558822a8dd541a48a551231817138ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:03:34.774724 22726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8dc1e34f64f34fb984bc15b7e4a73b67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:03:34.775720 23372 ts_tablet_manager.cc:1918] T 4c34798f99d24a489a05958d610fbffb P af341a2adada48adaa44eaa31c48cc05: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:03:34.777227 22929 tablet_service.cc:1467] Processing CreateTablet for tablet 8dc1e34f64f34fb984bc15b7e4a73b67 (DEFAULT_TABLE table=test-table [id=f558822a8dd541a48a551231817138ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:03:34.777601 22929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8dc1e34f64f34fb984bc15b7e4a73b67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:03:34.778206 23374 ts_tablet_manager.cc:1918] T 4c34798f99d24a489a05958d610fbffb P ae4c5de7483641ffb0fd3735fd550767: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:03:34.778394 23374 log.cc:1196] T 4c34798f99d24a489a05958d610fbffb P ae4c5de7483641ffb0fd3735fd550767: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-2/wal/wals/4c34798f99d24a489a05958d610fbffb
I20260811 12:03:34.778760 23372 log.cc:1196] T 4c34798f99d24a489a05958d610fbffb P af341a2adada48adaa44eaa31c48cc05: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-1/wal/wals/4c34798f99d24a489a05958d610fbffb
I20260811 12:03:34.778949 23374 ts_tablet_manager.cc:1939] T 4c34798f99d24a489a05958d610fbffb P ae4c5de7483641ffb0fd3735fd550767: Deleting consensus metadata
I20260811 12:03:34.779309 23372 ts_tablet_manager.cc:1939] T 4c34798f99d24a489a05958d610fbffb P af341a2adada48adaa44eaa31c48cc05: Deleting consensus metadata
I20260811 12:03:34.780144 22515 catalog_manager.cc:4841] TS ae4c5de7483641ffb0fd3735fd550767 (127.9.221.131:43229): tablet 4c34798f99d24a489a05958d610fbffb (table test-table [id=5a53782293dc4ce9a2cbe8753f3d2634]) successfully deleted
I20260811 12:03:34.780679 22513 catalog_manager.cc:4841] TS af341a2adada48adaa44eaa31c48cc05 (127.9.221.130:46333): tablet 4c34798f99d24a489a05958d610fbffb (table test-table [id=5a53782293dc4ce9a2cbe8753f3d2634]) successfully deleted
I20260811 12:03:34.781227 23142 tablet_service.cc:1467] Processing CreateTablet for tablet 8dc1e34f64f34fb984bc15b7e4a73b67 (DEFAULT_TABLE table=test-table [id=f558822a8dd541a48a551231817138ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:03:34.781678 23142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8dc1e34f64f34fb984bc15b7e4a73b67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:03:34.782666 23283 tablet_bootstrap.cc:492] T 8dc1e34f64f34fb984bc15b7e4a73b67 P 9265c12d3059486a8a74caa87ca611d4: Bootstrap starting.
I20260811 12:03:34.784526 23283 tablet_bootstrap.cc:654] T 8dc1e34f64f34fb984bc15b7e4a73b67 P 9265c12d3059486a8a74caa87ca611d4: Neither blocks nor log segments found. Creating new log.
I20260811 12:03:34.785616 23280 tablet_bootstrap.cc:492] T 8dc1e34f64f34fb984bc15b7e4a73b67 P af341a2adada48adaa44eaa31c48cc05: Bootstrap starting.
I20260811 12:03:34.787277 23280 tablet_bootstrap.cc:654] T 8dc1e34f64f34fb984bc15b7e4a73b67 P af341a2adada48adaa44eaa31c48cc05: Neither blocks nor log segments found. Creating new log.
I20260811 12:03:34.788938 23281 tablet_bootstrap.cc:492] T 8dc1e34f64f34fb984bc15b7e4a73b67 P ae4c5de7483641ffb0fd3735fd550767: Bootstrap starting.
I20260811 12:03:34.790943 23281 tablet_bootstrap.cc:654] T 8dc1e34f64f34fb984bc15b7e4a73b67 P ae4c5de7483641ffb0fd3735fd550767: Neither blocks nor log segments found. Creating new log.
I20260811 12:03:34.800801 23281 tablet_bootstrap.cc:492] T 8dc1e34f64f34fb984bc15b7e4a73b67 P ae4c5de7483641ffb0fd3735fd550767: No bootstrap required, opened a new log
I20260811 12:03:34.800853 23280 tablet_bootstrap.cc:492] T 8dc1e34f64f34fb984bc15b7e4a73b67 P af341a2adada48adaa44eaa31c48cc05: No bootstrap required, opened a new log
I20260811 12:03:34.800992 23281 ts_tablet_manager.cc:1397] T 8dc1e34f64f34fb984bc15b7e4a73b67 P ae4c5de7483641ffb0fd3735fd550767: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.009s
I20260811 12:03:34.801007 23280 ts_tablet_manager.cc:1397] T 8dc1e34f64f34fb984bc15b7e4a73b67 P af341a2adada48adaa44eaa31c48cc05: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.004s
I20260811 12:03:34.801080 23283 tablet_bootstrap.cc:492] T 8dc1e34f64f34fb984bc15b7e4a73b67 P 9265c12d3059486a8a74caa87ca611d4: No bootstrap required, opened a new log
I20260811 12:03:34.801262 23283 ts_tablet_manager.cc:1397] T 8dc1e34f64f34fb984bc15b7e4a73b67 P 9265c12d3059486a8a74caa87ca611d4: Time spent bootstrapping tablet: real 0.019s user 0.008s sys 0.003s
I20260811 12:03:34.801765 23280 raft_consensus.cc:348] T 8dc1e34f64f34fb984bc15b7e4a73b67 P af341a2adada48adaa44eaa31c48cc05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } }
I20260811 12:03:34.801851 23281 raft_consensus.cc:348] T 8dc1e34f64f34fb984bc15b7e4a73b67 P ae4c5de7483641ffb0fd3735fd550767 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } }
I20260811 12:03:34.802042 23280 raft_consensus.cc:374] T 8dc1e34f64f34fb984bc15b7e4a73b67 P af341a2adada48adaa44eaa31c48cc05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:03:34.802112 23281 raft_consensus.cc:374] T 8dc1e34f64f34fb984bc15b7e4a73b67 P ae4c5de7483641ffb0fd3735fd550767 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:03:34.802130 23280 raft_consensus.cc:729] T 8dc1e34f64f34fb984bc15b7e4a73b67 P af341a2adada48adaa44eaa31c48cc05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af341a2adada48adaa44eaa31c48cc05, State: Initialized, Role: FOLLOWER
I20260811 12:03:34.802201 23281 raft_consensus.cc:729] T 8dc1e34f64f34fb984bc15b7e4a73b67 P ae4c5de7483641ffb0fd3735fd550767 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae4c5de7483641ffb0fd3735fd550767, State: Initialized, Role: FOLLOWER
I20260811 12:03:34.802084 23283 raft_consensus.cc:348] T 8dc1e34f64f34fb984bc15b7e4a73b67 P 9265c12d3059486a8a74caa87ca611d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } }
I20260811 12:03:34.802380 23283 raft_consensus.cc:374] T 8dc1e34f64f34fb984bc15b7e4a73b67 P 9265c12d3059486a8a74caa87ca611d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:03:34.802469 23283 raft_consensus.cc:729] T 8dc1e34f64f34fb984bc15b7e4a73b67 P 9265c12d3059486a8a74caa87ca611d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9265c12d3059486a8a74caa87ca611d4, State: Initialized, Role: FOLLOWER
I20260811 12:03:34.802417 23280 consensus_queue.cc:260] T 8dc1e34f64f34fb984bc15b7e4a73b67 P af341a2adada48adaa44eaa31c48cc05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } }
I20260811 12:03:34.802762 23283 consensus_queue.cc:260] T 8dc1e34f64f34fb984bc15b7e4a73b67 P 9265c12d3059486a8a74caa87ca611d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } }
I20260811 12:03:34.802987 23280 ts_tablet_manager.cc:1428] T 8dc1e34f64f34fb984bc15b7e4a73b67 P af341a2adada48adaa44eaa31c48cc05: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260811 12:03:34.803310 23283 ts_tablet_manager.cc:1428] T 8dc1e34f64f34fb984bc15b7e4a73b67 P 9265c12d3059486a8a74caa87ca611d4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 12:03:34.802517 23281 consensus_queue.cc:260] T 8dc1e34f64f34fb984bc15b7e4a73b67 P ae4c5de7483641ffb0fd3735fd550767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } }
I20260811 12:03:34.806685 23281 ts_tablet_manager.cc:1428] T 8dc1e34f64f34fb984bc15b7e4a73b67 P ae4c5de7483641ffb0fd3735fd550767: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260811 12:03:34.840909 23323 raft_consensus.cc:482] T 8dc1e34f64f34fb984bc15b7e4a73b67 P ae4c5de7483641ffb0fd3735fd550767 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:03:34.841174 23323 raft_consensus.cc:504] T 8dc1e34f64f34fb984bc15b7e4a73b67 P ae4c5de7483641ffb0fd3735fd550767 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } }
I20260811 12:03:34.841892 23323 leader_election.cc:288] T 8dc1e34f64f34fb984bc15b7e4a73b67 P ae4c5de7483641ffb0fd3735fd550767 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9265c12d3059486a8a74caa87ca611d4 (127.9.221.129:38995), af341a2adada48adaa44eaa31c48cc05 (127.9.221.130:46333)
I20260811 12:03:34.842403 22763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8dc1e34f64f34fb984bc15b7e4a73b67" candidate_uuid: "ae4c5de7483641ffb0fd3735fd550767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9265c12d3059486a8a74caa87ca611d4" is_pre_election: true
I20260811 12:03:34.842689 22763 raft_consensus.cc:2454] T 8dc1e34f64f34fb984bc15b7e4a73b67 P 9265c12d3059486a8a74caa87ca611d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae4c5de7483641ffb0fd3735fd550767 in term 0.
I20260811 12:03:34.842682 22960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8dc1e34f64f34fb984bc15b7e4a73b67" candidate_uuid: "ae4c5de7483641ffb0fd3735fd550767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af341a2adada48adaa44eaa31c48cc05" is_pre_election: true
I20260811 12:03:34.842935 22960 raft_consensus.cc:2454] T 8dc1e34f64f34fb984bc15b7e4a73b67 P af341a2adada48adaa44eaa31c48cc05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae4c5de7483641ffb0fd3735fd550767 in term 0.
I20260811 12:03:34.843161 23064 leader_election.cc:302] T 8dc1e34f64f34fb984bc15b7e4a73b67 P ae4c5de7483641ffb0fd3735fd550767 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9265c12d3059486a8a74caa87ca611d4, ae4c5de7483641ffb0fd3735fd550767; no voters:
I20260811 12:03:34.843600 23323 raft_consensus.cc:2781] T 8dc1e34f64f34fb984bc15b7e4a73b67 P ae4c5de7483641ffb0fd3735fd550767 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:03:34.843731 23323 raft_consensus.cc:482] T 8dc1e34f64f34fb984bc15b7e4a73b67 P ae4c5de7483641ffb0fd3735fd550767 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:03:34.843858 23323 raft_consensus.cc:3037] T 8dc1e34f64f34fb984bc15b7e4a73b67 P ae4c5de7483641ffb0fd3735fd550767 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:03:34.846077 23323 raft_consensus.cc:504] T 8dc1e34f64f34fb984bc15b7e4a73b67 P ae4c5de7483641ffb0fd3735fd550767 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } }
I20260811 12:03:34.847154 22960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8dc1e34f64f34fb984bc15b7e4a73b67" candidate_uuid: "ae4c5de7483641ffb0fd3735fd550767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af341a2adada48adaa44eaa31c48cc05"
I20260811 12:03:34.847147 22763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8dc1e34f64f34fb984bc15b7e4a73b67" candidate_uuid: "ae4c5de7483641ffb0fd3735fd550767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9265c12d3059486a8a74caa87ca611d4"
I20260811 12:03:34.847357 22763 raft_consensus.cc:3037] T 8dc1e34f64f34fb984bc15b7e4a73b67 P 9265c12d3059486a8a74caa87ca611d4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:03:34.847352 22960 raft_consensus.cc:3037] T 8dc1e34f64f34fb984bc15b7e4a73b67 P af341a2adada48adaa44eaa31c48cc05 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:03:34.850363 22763 raft_consensus.cc:2454] T 8dc1e34f64f34fb984bc15b7e4a73b67 P 9265c12d3059486a8a74caa87ca611d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae4c5de7483641ffb0fd3735fd550767 in term 1.
I20260811 12:03:34.850373 22960 raft_consensus.cc:2454] T 8dc1e34f64f34fb984bc15b7e4a73b67 P af341a2adada48adaa44eaa31c48cc05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae4c5de7483641ffb0fd3735fd550767 in term 1.
I20260811 12:03:34.850831 23063 leader_election.cc:302] T 8dc1e34f64f34fb984bc15b7e4a73b67 P ae4c5de7483641ffb0fd3735fd550767 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae4c5de7483641ffb0fd3735fd550767, af341a2adada48adaa44eaa31c48cc05; no voters:
I20260811 12:03:34.851212 23288 raft_consensus.cc:2781] T 8dc1e34f64f34fb984bc15b7e4a73b67 P ae4c5de7483641ffb0fd3735fd550767 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:03:34.851373 23288 raft_consensus.cc:686] T 8dc1e34f64f34fb984bc15b7e4a73b67 P ae4c5de7483641ffb0fd3735fd550767 [term 1 LEADER]: Becoming Leader. State: Replica: ae4c5de7483641ffb0fd3735fd550767, State: Running, Role: LEADER
I20260811 12:03:34.851862 23288 consensus_queue.cc:237] T 8dc1e34f64f34fb984bc15b7e4a73b67 P ae4c5de7483641ffb0fd3735fd550767 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } }
I20260811 12:03:34.853085 23323 leader_election.cc:288] T 8dc1e34f64f34fb984bc15b7e4a73b67 P ae4c5de7483641ffb0fd3735fd550767 [CANDIDATE]: Term 1 election: Requested vote from peers 9265c12d3059486a8a74caa87ca611d4 (127.9.221.129:38995), af341a2adada48adaa44eaa31c48cc05 (127.9.221.130:46333)
I20260811 12:03:34.855332 22533 catalog_manager.cc:5495] T 8dc1e34f64f34fb984bc15b7e4a73b67 P ae4c5de7483641ffb0fd3735fd550767 reported cstate change: term changed from 0 to 1, leader changed from <none> to ae4c5de7483641ffb0fd3735fd550767 (127.9.221.131). New cstate: current_term: 1 leader_uuid: "ae4c5de7483641ffb0fd3735fd550767" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae4c5de7483641ffb0fd3735fd550767" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43229 } health_report { overall_health: HEALTHY } } }
I20260811 12:03:34.889986 22763 raft_consensus.cc:1261] T 8dc1e34f64f34fb984bc15b7e4a73b67 P 9265c12d3059486a8a74caa87ca611d4 [term 1 FOLLOWER]: Refusing update from remote peer ae4c5de7483641ffb0fd3735fd550767: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:03:34.889991 22960 raft_consensus.cc:1261] T 8dc1e34f64f34fb984bc15b7e4a73b67 P af341a2adada48adaa44eaa31c48cc05 [term 1 FOLLOWER]: Refusing update from remote peer ae4c5de7483641ffb0fd3735fd550767: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:03:34.890609 23288 consensus_queue.cc:1035] T 8dc1e34f64f34fb984bc15b7e4a73b67 P ae4c5de7483641ffb0fd3735fd550767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9265c12d3059486a8a74caa87ca611d4" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:03:34.890988 23323 consensus_queue.cc:1035] T 8dc1e34f64f34fb984bc15b7e4a73b67 P ae4c5de7483641ffb0fd3735fd550767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af341a2adada48adaa44eaa31c48cc05" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:03:34.970497 22533 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:03:34.970767 22533 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:03:34.973750 22533 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bac306678bd5437a8b1101dae6b4b615: Sending DeleteTablet for 3 replicas of tablet 8dc1e34f64f34fb984bc15b7e4a73b67
I20260811 12:03:34.974498 22726 tablet_service.cc:1514] Processing DeleteTablet for tablet 8dc1e34f64f34fb984bc15b7e4a73b67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:03:34 UTC) from {username='oryx', principal='oryx/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:33434
I20260811 12:03:34.974658 22929 tablet_service.cc:1514] Processing DeleteTablet for tablet 8dc1e34f64f34fb984bc15b7e4a73b67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:03:34 UTC) from {username='oryx', principal='oryx/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:59724
I20260811 12:03:34.974802 23373 tablet_replica.cc:330] T 8dc1e34f64f34fb984bc15b7e4a73b67 P 9265c12d3059486a8a74caa87ca611d4: stopping tablet replica
I20260811 12:03:34.974972 23372 tablet_replica.cc:330] T 8dc1e34f64f34fb984bc15b7e4a73b67 P af341a2adada48adaa44eaa31c48cc05: stopping tablet replica
I20260811 12:03:34.974992 23373 raft_consensus.cc:2229] T 8dc1e34f64f34fb984bc15b7e4a73b67 P 9265c12d3059486a8a74caa87ca611d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:03:34.975170 23372 raft_consensus.cc:2229] T 8dc1e34f64f34fb984bc15b7e4a73b67 P af341a2adada48adaa44eaa31c48cc05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:03:34.975198 23373 raft_consensus.cc:2258] T 8dc1e34f64f34fb984bc15b7e4a73b67 P 9265c12d3059486a8a74caa87ca611d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:03:34.975368 23372 raft_consensus.cc:2258] T 8dc1e34f64f34fb984bc15b7e4a73b67 P af341a2adada48adaa44eaa31c48cc05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:03:34.975961 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 22639
I20260811 12:03:34.975996 23142 tablet_service.cc:1514] Processing DeleteTablet for tablet 8dc1e34f64f34fb984bc15b7e4a73b67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:03:34 UTC) from {username='oryx', principal='oryx/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:55462
I20260811 12:03:34.976315 23374 tablet_replica.cc:330] T 8dc1e34f64f34fb984bc15b7e4a73b67 P ae4c5de7483641ffb0fd3735fd550767: stopping tablet replica
I20260811 12:03:34.976522 23374 raft_consensus.cc:2229] T 8dc1e34f64f34fb984bc15b7e4a73b67 P ae4c5de7483641ffb0fd3735fd550767 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:03:34.976863 23374 raft_consensus.cc:2258] T 8dc1e34f64f34fb984bc15b7e4a73b67 P ae4c5de7483641ffb0fd3735fd550767 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:03:34.977069 23373 ts_tablet_manager.cc:1905] T 8dc1e34f64f34fb984bc15b7e4a73b67 P 9265c12d3059486a8a74caa87ca611d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:03:34.978662 23372 ts_tablet_manager.cc:1905] T 8dc1e34f64f34fb984bc15b7e4a73b67 P af341a2adada48adaa44eaa31c48cc05: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:03:34.981776 23374 ts_tablet_manager.cc:1905] T 8dc1e34f64f34fb984bc15b7e4a73b67 P ae4c5de7483641ffb0fd3735fd550767: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:03:34.986275 23373 ts_tablet_manager.cc:1918] T 8dc1e34f64f34fb984bc15b7e4a73b67 P 9265c12d3059486a8a74caa87ca611d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:03:34.986452 23373 log.cc:1196] T 8dc1e34f64f34fb984bc15b7e4a73b67 P 9265c12d3059486a8a74caa87ca611d4: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-0/wal/wals/8dc1e34f64f34fb984bc15b7e4a73b67
I20260811 12:03:34.986996 23373 ts_tablet_manager.cc:1939] T 8dc1e34f64f34fb984bc15b7e4a73b67 P 9265c12d3059486a8a74caa87ca611d4: Deleting consensus metadata
I20260811 12:03:34.991051 23372 ts_tablet_manager.cc:1918] T 8dc1e34f64f34fb984bc15b7e4a73b67 P af341a2adada48adaa44eaa31c48cc05: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:03:34.991232 23372 log.cc:1196] T 8dc1e34f64f34fb984bc15b7e4a73b67 P af341a2adada48adaa44eaa31c48cc05: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-1/wal/wals/8dc1e34f64f34fb984bc15b7e4a73b67
I20260811 12:03:34.991743 23372 ts_tablet_manager.cc:1939] T 8dc1e34f64f34fb984bc15b7e4a73b67 P af341a2adada48adaa44eaa31c48cc05: Deleting consensus metadata
I20260811 12:03:34.992957 22513 catalog_manager.cc:4841] TS af341a2adada48adaa44eaa31c48cc05 (127.9.221.130:46333): tablet 8dc1e34f64f34fb984bc15b7e4a73b67 (table test-table [id=f558822a8dd541a48a551231817138ea]) successfully deleted
I20260811 12:03:34.993729 23374 ts_tablet_manager.cc:1918] T 8dc1e34f64f34fb984bc15b7e4a73b67 P ae4c5de7483641ffb0fd3735fd550767: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:03:34.993903 23374 log.cc:1196] T 8dc1e34f64f34fb984bc15b7e4a73b67 P ae4c5de7483641ffb0fd3735fd550767: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449743084027-10102-0/minicluster-data/ts-2/wal/wals/8dc1e34f64f34fb984bc15b7e4a73b67
I20260811 12:03:34.994400 23374 ts_tablet_manager.cc:1939] T 8dc1e34f64f34fb984bc15b7e4a73b67 P ae4c5de7483641ffb0fd3735fd550767: Deleting consensus metadata
I20260811 12:03:34.995539 22515 catalog_manager.cc:4841] TS ae4c5de7483641ffb0fd3735fd550767 (127.9.221.131:43229): tablet 8dc1e34f64f34fb984bc15b7e4a73b67 (table test-table [id=f558822a8dd541a48a551231817138ea]) successfully deleted
I20260811 12:03:34.995816 22515 catalog_manager.cc:4841] TS 9265c12d3059486a8a74caa87ca611d4 (127.9.221.129:38995): tablet 8dc1e34f64f34fb984bc15b7e4a73b67 (table test-table [id=f558822a8dd541a48a551231817138ea]) successfully deleted
Aug 11 12:03:34 dist-test-slave-mjjr krb5kdc[22459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449814, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.221.129@KRBTEST.COM
I20260811 12:03:35.009083 23410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:34.977614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33462 (local address 127.9.221.129:38995)
0811 12:03:34.991673 (+ 14059us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:34.991679 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:34.991700 (+ 21us) server_negotiation.cc:407] Connection header received
0811 12:03:34.991753 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:34.991759 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:34.991899 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:34.993765 (+ 1866us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:34.993997 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:34.995109 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:34.997024 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:34.997466 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:34.997561 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:35.002160 (+ 4599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:35.002180 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:35.002200 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:35.002245 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:35.006083 (+ 3838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:35.006782 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:35.006793 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:35.006797 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:35.006966 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:35.007327 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:35.007332 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:35.007336 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:35.007890 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:35.008207 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:35.008405 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 12:03:35.008639 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13905,"thread_start_us":17615,"threads_started":1}
I20260811 12:03:35.009807 22822 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:03:35.134701 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 22639
I20260811 12:03:35.166261 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 22834
Aug 11 12:03:35 dist-test-slave-mjjr krb5kdc[22459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449814, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.221.130@KRBTEST.COM
I20260811 12:03:35.180891 23414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:35.167115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59734 (local address 127.9.221.130:46333)
0811 12:03:35.167695 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:35.167700 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:35.167717 (+ 17us) server_negotiation.cc:407] Connection header received
0811 12:03:35.167774 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:35.167778 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:35.167846 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:35.167953 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:35.169246 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:35.169925 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:35.171174 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:35.171613 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:35.171812 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:35.175905 (+ 4093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:35.175918 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:35.175933 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:35.175965 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:35.178575 (+ 2610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:35.179120 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:35.179126 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:35.179129 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:35.179251 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:35.179546 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:35.179549 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:35.179551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:35.179930 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:35.180145 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:35.180390 (+ 245us) server_negotiation.cc:299] Negotiation successful
0811 12:03:35.180616 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":291,"threads_started":1}
I20260811 12:03:35.181730 23030 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:03:35.304184 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 22834
I20260811 12:03:35.338534 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 23041
Aug 11 12:03:35 dist-test-slave-mjjr krb5kdc[22459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449814, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.221.131@KRBTEST.COM
I20260811 12:03:35.353300 23416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:35.339328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55474 (local address 127.9.221.131:43229)
0811 12:03:35.339850 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:35.339854 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:35.339871 (+ 17us) server_negotiation.cc:407] Connection header received
0811 12:03:35.339927 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:35.339930 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:35.339991 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:35.340083 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:35.341365 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:35.342033 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:35.343281 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:35.343794 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:35.343885 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:35.348077 (+ 4192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:35.348089 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:35.348105 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:35.348138 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:35.350848 (+ 2710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:35.351446 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:35.351463 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:35.351465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:35.351607 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:35.351985 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:35.351988 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:35.351990 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:35.352348 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:35.352576 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:35.352775 (+ 199us) server_negotiation.cc:299] Negotiation successful
0811 12:03:35.352929 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":254,"threads_started":1}
I20260811 12:03:35.353926 23237 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:03:35.478477 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 23041
I20260811 12:03:35.505527 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 22481
I20260811 12:03:35.506767 22591 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:03:35.627725 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 22481
2026-08-11T12:03:35Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6962 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786449743084027-10102-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:03:35 dist-test-slave-mjjr krb5kdc[23422](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:03:35 dist-test-slave-mjjr krb5kdc[23422](info): set up 2 sockets
Aug 11 12:03:35 dist-test-slave-mjjr krb5kdc[23422](info): commencing operation
krb5kdc: starting...
W20260811 12:03:36.944902 10102 mini_kdc.cc:121] Time spent starting KDC: real 1.260s user 0.002s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Aug 11 12:03:37 dist-test-slave-mjjr krb5kdc[23422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449817, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:03:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:03:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.221.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
I20260811 12:03:37.181191 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:36929
--webserver_interface=127.9.221.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:32809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:36929
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:03:37.532912 23440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:03:37.533234 23440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:03:37.533389 23440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:03:37.540591 23440 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:03:37.540730 23440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:03:37.540843 23440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:03:37.540938 23440 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:03:37.541028 23440 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:03:37.550709 23440 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:32809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:36929
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:36929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.221.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:03:37.552116 23440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:03:37.553434 23440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:03:37.565645 23449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:37.565724 23455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:37.565682 23450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:03:37.566443 23440 server_base.cc:1034] running on GCE node
I20260811 12:03:37.567205 23440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:03:37.568590 23440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:03:37.569823 23440 hybrid_clock.cc:648] HybridClock initialized: now 1786449817569782 us; error 52 us; skew 500 ppm
I20260811 12:03:37.570170 23440 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-11T12:03:37Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (2003 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:03:37 dist-test-slave-mjjr krb5kdc[23464](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:03:37 dist-test-slave-mjjr krb5kdc[23464](info): set up 2 sockets
Aug 11 12:03:37 dist-test-slave-mjjr krb5kdc[23464](info): commencing operation
krb5kdc: starting...
W20260811 12:03:38.943341 10102 mini_kdc.cc:121] Time spent starting KDC: real 1.255s user 0.001s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Aug 11 12:03:39 dist-test-slave-mjjr krb5kdc[23464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:03:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:03:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.221.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
I20260811 12:03:39.188181 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:44917
--webserver_interface=127.9.221.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:33157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:44917
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:03:39.519284 23482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:03:39.519584 23482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:03:39.519796 23482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:03:39.526773 23482 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:03:39.526906 23482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:03:39.527007 23482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:03:39.527107 23482 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:03:39.527190 23482 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:03:39.536679 23482 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:33157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:44917
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:44917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.221.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:03:39.537957 23482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:03:39.539274 23482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:03:39.550841 23488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:39.551002 23491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:39.550851 23494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:03:39.551963 23482 server_base.cc:1034] running on GCE node
I20260811 12:03:39.552701 23482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:03:39.554004 23482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:03:39.555248 23482 hybrid_clock.cc:648] HybridClock initialized: now 1786449819555193 us; error 46 us; skew 500 ppm
I20260811 12:03:39.555609 23482 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:03:39 dist-test-slave-mjjr krb5kdc[23464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449819, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:03:39.558626 23482 init.cc:377] Logged in from keytab as kudu/127.9.221.190@KRBTEST.COM (short username kudu)
I20260811 12:03:39.560010 23482 webserver.cc:460] Webserver started at http://127.9.221.190:39137/ using document root <none> and password file <none>
I20260811 12:03:39.560660 23482 fs_manager.cc:359] Metadata directory not provided
I20260811 12:03:39.560824 23482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:03:39.561126 23482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:03:39.563983 23482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "04d4ec21ea264036a2951c2979a0d3b4"
format_stamp: "Formatted at 2026-08-11 12:03:39 on dist-test-slave-mjjr"
server_key: "ba6dd68ee5c64d5318c1f33d0ddd3208"
server_key_iv: "c5cd7e4784e78b8545958d912b67acf5"
server_key_version: "encryptionkey@0"
I20260811 12:03:39.564715 23482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "04d4ec21ea264036a2951c2979a0d3b4"
format_stamp: "Formatted at 2026-08-11 12:03:39 on dist-test-slave-mjjr"
server_key: "ba6dd68ee5c64d5318c1f33d0ddd3208"
server_key_iv: "c5cd7e4784e78b8545958d912b67acf5"
server_key_version: "encryptionkey@0"
I20260811 12:03:39.569943 23482 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:03:39.573421 23505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:39.574673 23482 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 12:03:39.574853 23482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "04d4ec21ea264036a2951c2979a0d3b4"
format_stamp: "Formatted at 2026-08-11 12:03:39 on dist-test-slave-mjjr"
server_key: "ba6dd68ee5c64d5318c1f33d0ddd3208"
server_key_iv: "c5cd7e4784e78b8545958d912b67acf5"
server_key_version: "encryptionkey@0"
I20260811 12:03:39.575037 23482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:03:39.601219 23482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:03:39.605403 23482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:03:39.605724 23482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:03:39.625562 23482 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.190:44917
I20260811 12:03:39.625567 23578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.190:44917 every 8 connection(s)
I20260811 12:03:39.626967 23482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:03:39.630049 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 23482
I20260811 12:03:39.630306 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:03:39.630926 23580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:03:39.630947 10102 external_mini_cluster.cc:1428] Setting key 9047fca4cfec677932ebd91727f71822
I20260811 12:03:39.645982 23580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04d4ec21ea264036a2951c2979a0d3b4: Bootstrap starting.
Aug 11 12:03:39 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:03:39.650321 23580 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04d4ec21ea264036a2951c2979a0d3b4: Neither blocks nor log segments found. Creating new log.
I20260811 12:03:39.651798 23580 log.cc:824] T 00000000000000000000000000000000 P 04d4ec21ea264036a2951c2979a0d3b4: Log is configured to *not* fsync() on all Append() calls
I20260811 12:03:39.655207 23580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04d4ec21ea264036a2951c2979a0d3b4: No bootstrap required, opened a new log
I20260811 12:03:39.656227 23583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:39.632856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45934 (local address 127.9.221.190:44917)
0811 12:03:39.633718 (+ 862us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:39.633734 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:39.633787 (+ 53us) server_negotiation.cc:407] Connection header received
0811 12:03:39.636076 (+ 2289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:39.636102 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:39.636582 (+ 480us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:39.637016 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:39.638345 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:39.640468 (+ 2123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:39.641756 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:39.642933 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:39.643148 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:39.647134 (+ 3986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:39.647145 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:39.647164 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:39.647234 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:39.651505 (+ 4271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:39.652282 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:39.652296 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:39.652314 (+ 18us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:39.652639 (+ 325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:39.653170 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:39.653175 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:39.653212 (+ 37us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:39.653871 (+ 659us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:39.654188 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:39.654528 (+ 340us) server_negotiation.cc:299] Negotiation successful
0811 12:03:39.654935 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":352,"threads_started":1}
I20260811 12:03:39.662693 23580 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04d4ec21ea264036a2951c2979a0d3b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04d4ec21ea264036a2951c2979a0d3b4" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 44917 } }
I20260811 12:03:39.663007 23580 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04d4ec21ea264036a2951c2979a0d3b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:03:39.663087 23580 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04d4ec21ea264036a2951c2979a0d3b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04d4ec21ea264036a2951c2979a0d3b4, State: Initialized, Role: FOLLOWER
I20260811 12:03:39.663730 23580 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04d4ec21ea264036a2951c2979a0d3b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04d4ec21ea264036a2951c2979a0d3b4" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 44917 } }
I20260811 12:03:39.663998 23580 raft_consensus.cc:388] T 00000000000000000000000000000000 P 04d4ec21ea264036a2951c2979a0d3b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:03:39.664148 23580 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04d4ec21ea264036a2951c2979a0d3b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:03:39.664331 23580 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04d4ec21ea264036a2951c2979a0d3b4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:03:39.666424 23580 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04d4ec21ea264036a2951c2979a0d3b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04d4ec21ea264036a2951c2979a0d3b4" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 44917 } }
I20260811 12:03:39.666841 23580 leader_election.cc:302] T 00000000000000000000000000000000 P 04d4ec21ea264036a2951c2979a0d3b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04d4ec21ea264036a2951c2979a0d3b4; no voters:
I20260811 12:03:39.667348 23580 leader_election.cc:288] T 00000000000000000000000000000000 P 04d4ec21ea264036a2951c2979a0d3b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:03:39.667488 23586 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04d4ec21ea264036a2951c2979a0d3b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:03:39.668193 23586 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04d4ec21ea264036a2951c2979a0d3b4 [term 1 LEADER]: Becoming Leader. State: Replica: 04d4ec21ea264036a2951c2979a0d3b4, State: Running, Role: LEADER
I20260811 12:03:39.668764 23586 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04d4ec21ea264036a2951c2979a0d3b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04d4ec21ea264036a2951c2979a0d3b4" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 44917 } }
I20260811 12:03:39.669281 23580 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04d4ec21ea264036a2951c2979a0d3b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:03:39.671769 23587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04d4ec21ea264036a2951c2979a0d3b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "04d4ec21ea264036a2951c2979a0d3b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04d4ec21ea264036a2951c2979a0d3b4" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 44917 } } }
I20260811 12:03:39.672114 23587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04d4ec21ea264036a2951c2979a0d3b4 [sys.catalog]: This master's current role is: LEADER
I20260811 12:03:39.672657 23588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04d4ec21ea264036a2951c2979a0d3b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04d4ec21ea264036a2951c2979a0d3b4. Latest consensus state: current_term: 1 leader_uuid: "04d4ec21ea264036a2951c2979a0d3b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04d4ec21ea264036a2951c2979a0d3b4" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 44917 } } }
I20260811 12:03:39.673038 23588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04d4ec21ea264036a2951c2979a0d3b4 [sys.catalog]: This master's current role is: LEADER
I20260811 12:03:39.676832 23596 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:03:39.682313 23596 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:03:39.690651 23596 catalog_manager.cc:1347] Generated new cluster ID: 18026015fd3b4cdd96912bbad3d82074
I20260811 12:03:39.690778 23596 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:03:39.712752 23596 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:03:39.714599 23596 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:03:39.731128 23596 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 04d4ec21ea264036a2951c2979a0d3b4: Generated new TSK 0
I20260811 12:03:39.732192 23596 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.221.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab.
Entry for principal kudu/127.9.221.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab.
I20260811 12:03:39.799050 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.129:0
--local_ip_for_outbound_sockets=127.9.221.129
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:44917
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab
--principal=kudu/127.9.221.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:33157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:03:40.134188 23614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:03:40.134485 23614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:03:40.134644 23614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:03:40.141667 23614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:03:40.141809 23614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:03:40.141921 23614 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.129
I20260811 12:03:40.151335 23614 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:33157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab
--principal=kudu/127.9.221.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:44917
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.129
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:03:40.152634 23614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:03:40.153873 23614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:03:40.166733 23624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:40.166741 23625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:40.166889 23627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:03:40.167624 23614 server_base.cc:1034] running on GCE node
I20260811 12:03:40.168182 23614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:03:40.169026 23614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:03:40.170238 23614 hybrid_clock.cc:648] HybridClock initialized: now 1786449820170172 us; error 58 us; skew 500 ppm
I20260811 12:03:40.170496 23614 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:03:40 dist-test-slave-mjjr krb5kdc[23464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449820, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:03:40.173663 23614 init.cc:377] Logged in from keytab as kudu/127.9.221.129@KRBTEST.COM (short username kudu)
I20260811 12:03:40.175010 23614 webserver.cc:460] Webserver started at http://127.9.221.129:43379/ using document root <none> and password file <none>
I20260811 12:03:40.175586 23614 fs_manager.cc:359] Metadata directory not provided
I20260811 12:03:40.175762 23614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:03:40.176067 23614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:03:40.178989 23614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-0/data/instance:
uuid: "9b199daf7b5d4f1ebab0647ca447aad2"
format_stamp: "Formatted at 2026-08-11 12:03:40 on dist-test-slave-mjjr"
server_key: "dc480eb3c875339ec3221b2de5a13494"
server_key_iv: "769355e45cfcff6d9cf87a8584383092"
server_key_version: "encryptionkey@0"
I20260811 12:03:40.179824 23614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance:
uuid: "9b199daf7b5d4f1ebab0647ca447aad2"
format_stamp: "Formatted at 2026-08-11 12:03:40 on dist-test-slave-mjjr"
server_key: "dc480eb3c875339ec3221b2de5a13494"
server_key_iv: "769355e45cfcff6d9cf87a8584383092"
server_key_version: "encryptionkey@0"
I20260811 12:03:40.184602 23614 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 12:03:40.187704 23635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:40.188916 23614 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 12:03:40.189076 23614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-0/wal
uuid: "9b199daf7b5d4f1ebab0647ca447aad2"
format_stamp: "Formatted at 2026-08-11 12:03:40 on dist-test-slave-mjjr"
server_key: "dc480eb3c875339ec3221b2de5a13494"
server_key_iv: "769355e45cfcff6d9cf87a8584383092"
server_key_version: "encryptionkey@0"
I20260811 12:03:40.189248 23614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:03:40.201663 23614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:03:40.205204 23614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:03:40.205480 23614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:03:40.206317 23614 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:03:40.207517 23614 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:03:40.207588 23614 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:40.207782 23614 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:03:40.207842 23614 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:40.242177 23614 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.129:33255
I20260811 12:03:40.242226 23800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.129:33255 every 8 connection(s)
I20260811 12:03:40.243397 23614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
I20260811 12:03:40.253621 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 23614
I20260811 12:03:40.253854 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance
I20260811 12:03:40.254606 10102 external_mini_cluster.cc:1428] Setting key f6622499e25f19b4e9083107cf8b1ebe
Aug 11 12:03:40 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449820, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:03:40.264051 23804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:40.245847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:48273 (local address 127.9.221.190:44917)
0811 12:03:40.246304 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:40.246309 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:40.247152 (+ 843us) server_negotiation.cc:407] Connection header received
0811 12:03:40.249022 (+ 1870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:40.249027 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:40.249125 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:40.249236 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:40.250646 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:40.251369 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:40.253269 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:40.253974 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:40.254128 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:40.258360 (+ 4232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:40.258365 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:40.258368 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:40.258399 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:40.260877 (+ 2478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:40.261771 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:40.261775 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:40.261777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:40.261901 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:40.262384 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:40.262387 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:40.262389 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:40.262619 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:40.262722 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:40.263579 (+ 857us) server_negotiation.cc:299] Negotiation successful
0811 12:03:40.263788 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":259,"threads_started":1}
I20260811 12:03:40.265460 23805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:40.246314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.129:48273)
0811 12:03:40.246997 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:40.247040 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:40.248756 (+ 1716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:40.249449 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:40.249456 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:40.249957 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:40.250490 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:40.250506 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:40.251527 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:40.251531 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:40.253039 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:40.253051 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:40.254184 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:40.254189 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:40.254398 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:40.255105 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:40.255150 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:40.258151 (+ 3001us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:40.261011 (+ 2860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:40.261021 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:40.261037 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:40.261659 (+ 622us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:40.262016 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:40.262023 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:40.262029 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:40.262256 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:40.262733 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:40.262742 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:40.263166 (+ 424us) client_negotiation.cc:769] Sending connection context
0811 12:03:40.263487 (+ 321us) client_negotiation.cc:240] Negotiation successful
0811 12:03:40.264060 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":279,"threads_started":1}
I20260811 12:03:40.266973 23802 heartbeater.cc:347] Connected to a master server at 127.9.221.190:44917
I20260811 12:03:40.267257 23802 heartbeater.cc:464] Registering TS with master...
I20260811 12:03:40.267994 23802 heartbeater.cc:511] Master 127.9.221.190:44917 requested a full tablet report, sending...
I20260811 12:03:40.270037 23528 ts_manager.cc:194] Registered new tserver with Master: 9b199daf7b5d4f1ebab0647ca447aad2 (127.9.221.129:33255)
I20260811 12:03:40.271489 23528 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.129@KRBTEST.COM'} at 127.9.221.129:48273
WARNING: no policy specified for kudu/127.9.221.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab.
Entry for principal kudu/127.9.221.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab.
I20260811 12:03:40.308588 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.130:0
--local_ip_for_outbound_sockets=127.9.221.130
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:44917
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab
--principal=kudu/127.9.221.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:33157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:03:40.656271 23808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:03:40.656595 23808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:03:40.656791 23808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:03:40.664167 23808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:03:40.664316 23808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:03:40.664494 23808 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.130
I20260811 12:03:40.674898 23808 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:33157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab
--principal=kudu/127.9.221.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:44917
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.130
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:03:40.676237 23808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:03:40.677613 23808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:03:40.690914 23817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:40.690965 23820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:40.691002 23818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:03:40.692164 23808 server_base.cc:1034] running on GCE node
I20260811 12:03:40.692831 23808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:03:40.693873 23808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:03:40.695108 23808 hybrid_clock.cc:648] HybridClock initialized: now 1786449820695068 us; error 54 us; skew 500 ppm
I20260811 12:03:40.695401 23808 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:03:40 dist-test-slave-mjjr krb5kdc[23464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449820, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:03:40.698726 23808 init.cc:377] Logged in from keytab as kudu/127.9.221.130@KRBTEST.COM (short username kudu)
I20260811 12:03:40.700170 23808 webserver.cc:460] Webserver started at http://127.9.221.130:41737/ using document root <none> and password file <none>
I20260811 12:03:40.700850 23808 fs_manager.cc:359] Metadata directory not provided
I20260811 12:03:40.700989 23808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:03:40.701278 23808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:03:40.704092 23808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-1/data/instance:
uuid: "747455b70d10478fb9d430e4638c59ab"
format_stamp: "Formatted at 2026-08-11 12:03:40 on dist-test-slave-mjjr"
server_key: "500c4996b253020f1baaa677dbeceb61"
server_key_iv: "fbd5e290352294ce1a614981b138a677"
server_key_version: "encryptionkey@0"
I20260811 12:03:40.704870 23808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance:
uuid: "747455b70d10478fb9d430e4638c59ab"
format_stamp: "Formatted at 2026-08-11 12:03:40 on dist-test-slave-mjjr"
server_key: "500c4996b253020f1baaa677dbeceb61"
server_key_iv: "fbd5e290352294ce1a614981b138a677"
server_key_version: "encryptionkey@0"
I20260811 12:03:40.709798 23808 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:03:40.713081 23832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:40.714332 23808 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 12:03:40.714512 23808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-1/wal
uuid: "747455b70d10478fb9d430e4638c59ab"
format_stamp: "Formatted at 2026-08-11 12:03:40 on dist-test-slave-mjjr"
server_key: "500c4996b253020f1baaa677dbeceb61"
server_key_iv: "fbd5e290352294ce1a614981b138a677"
server_key_version: "encryptionkey@0"
I20260811 12:03:40.714718 23808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:03:40.730443 23808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:03:40.734131 23808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:03:40.734409 23808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:03:40.735296 23808 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:03:40.736595 23808 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:03:40.736693 23808 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:40.736858 23808 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:03:40.736941 23808 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:40.771149 23808 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.130:41735
I20260811 12:03:40.771183 23999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.130:41735 every 8 connection(s)
I20260811 12:03:40.772550 23808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
I20260811 12:03:40.780366 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 23808
I20260811 12:03:40.780603 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance
I20260811 12:03:40.781183 10102 external_mini_cluster.cc:1428] Setting key 7a2663bc9879282531808c5df1c6c14b
Aug 11 12:03:40 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449820, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.130@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:03:40.796671 24004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:40.775284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:60983 (local address 127.9.221.190:44917)
0811 12:03:40.775809 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:40.775814 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:40.776847 (+ 1033us) server_negotiation.cc:407] Connection header received
0811 12:03:40.778899 (+ 2052us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:40.778905 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:40.779001 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:40.779117 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:40.781099 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:40.782012 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:40.784053 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:40.784499 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:40.784598 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:40.789262 (+ 4664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:40.789270 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:40.789277 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:40.789323 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:40.793060 (+ 3737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:40.794116 (+ 1056us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:40.794122 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:40.794124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:40.794271 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:40.794736 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:40.794740 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:40.794742 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:40.795063 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:40.795223 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:40.796012 (+ 789us) server_negotiation.cc:299] Negotiation successful
0811 12:03:40.796255 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":274,"threads_started":1}
I20260811 12:03:40.797760 24005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:40.775813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.130:60983)
0811 12:03:40.776668 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:40.776728 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:40.778611 (+ 1883us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:40.779367 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:40.779375 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:40.780028 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:40.780907 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:40.780932 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:40.782186 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:40.782193 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:40.783885 (+ 1692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:40.783901 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:40.784734 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:40.784745 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:40.785109 (+ 364us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:40.785868 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:40.785904 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:40.789003 (+ 3099us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:40.793205 (+ 4202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:40.793217 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:40.793237 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:40.793961 (+ 724us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:40.794394 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:40.794399 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:40.794402 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:40.794599 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:40.795194 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:40.795202 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:40.795583 (+ 381us) client_negotiation.cc:769] Sending connection context
0811 12:03:40.795912 (+ 329us) client_negotiation.cc:240] Negotiation successful
0811 12:03:40.796502 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":430,"threads_started":1}
I20260811 12:03:40.799285 24000 heartbeater.cc:347] Connected to a master server at 127.9.221.190:44917
I20260811 12:03:40.799604 24000 heartbeater.cc:464] Registering TS with master...
I20260811 12:03:40.800379 24000 heartbeater.cc:511] Master 127.9.221.190:44917 requested a full tablet report, sending...
I20260811 12:03:40.801880 23528 ts_manager.cc:194] Registered new tserver with Master: 747455b70d10478fb9d430e4638c59ab (127.9.221.130:41735)
I20260811 12:03:40.802706 23528 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.130@KRBTEST.COM'} at 127.9.221.130:60983
WARNING: no policy specified for kudu/127.9.221.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab.
Entry for principal kudu/127.9.221.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab.
I20260811 12:03:40.838992 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.131:0
--local_ip_for_outbound_sockets=127.9.221.131
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:44917
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab
--principal=kudu/127.9.221.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:33157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:03:41.169445 24016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:03:41.169754 24016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:03:41.169852 24016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:03:41.177419 24016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:03:41.177518 24016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:03:41.177634 24016 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.131
I20260811 12:03:41.187192 24016 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:33157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab
--principal=kudu/127.9.221.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:44917
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.131
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:03:41.188405 24016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:03:41.189622 24016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:03:41.202515 24022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:41.202515 24025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:41.202561 24023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:03:41.203805 24016 server_base.cc:1034] running on GCE node
I20260811 12:03:41.204450 24016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:03:41.205329 24016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:03:41.206554 24016 hybrid_clock.cc:648] HybridClock initialized: now 1786449821206507 us; error 51 us; skew 500 ppm
I20260811 12:03:41.206854 24016 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:03:41 dist-test-slave-mjjr krb5kdc[23464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449821, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:03:41.209880 24016 init.cc:377] Logged in from keytab as kudu/127.9.221.131@KRBTEST.COM (short username kudu)
I20260811 12:03:41.211236 24016 webserver.cc:460] Webserver started at http://127.9.221.131:41007/ using document root <none> and password file <none>
I20260811 12:03:41.212005 24016 fs_manager.cc:359] Metadata directory not provided
I20260811 12:03:41.212159 24016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:03:41.212576 24016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:03:41.215415 24016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-2/data/instance:
uuid: "bafb533e748949028cf330ffd40257d7"
format_stamp: "Formatted at 2026-08-11 12:03:41 on dist-test-slave-mjjr"
server_key: "9e46f57954cc07370be0affe3e92879c"
server_key_iv: "76c8193359608690559d688b9825aab4"
server_key_version: "encryptionkey@0"
I20260811 12:03:41.216276 24016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance:
uuid: "bafb533e748949028cf330ffd40257d7"
format_stamp: "Formatted at 2026-08-11 12:03:41 on dist-test-slave-mjjr"
server_key: "9e46f57954cc07370be0affe3e92879c"
server_key_iv: "76c8193359608690559d688b9825aab4"
server_key_version: "encryptionkey@0"
I20260811 12:03:41.221510 24016 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:03:41.224767 24032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:41.226006 24016 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:03:41.226163 24016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-2/wal
uuid: "bafb533e748949028cf330ffd40257d7"
format_stamp: "Formatted at 2026-08-11 12:03:41 on dist-test-slave-mjjr"
server_key: "9e46f57954cc07370be0affe3e92879c"
server_key_iv: "76c8193359608690559d688b9825aab4"
server_key_version: "encryptionkey@0"
I20260811 12:03:41.226346 24016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:03:41.250144 24016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:03:41.253744 24016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:03:41.254035 24016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:03:41.254935 24016 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:03:41.256253 24016 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:03:41.256326 24016 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:41.256489 24016 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:03:41.256541 24016 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:41.274231 23802 heartbeater.cc:503] Master 127.9.221.190:44917 was elected leader, sending a full tablet report...
I20260811 12:03:41.289479 24016 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.131:43925
I20260811 12:03:41.289561 24194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.131:43925 every 8 connection(s)
I20260811 12:03:41.290704 24016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
I20260811 12:03:41.293747 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 24016
I20260811 12:03:41.293936 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance
I20260811 12:03:41.294389 10102 external_mini_cluster.cc:1428] Setting key b46cdf537ee62d1d21ca85d414b8adb6
Aug 11 12:03:41 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449821, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.131@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:03:41.310398 24004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:41.293138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:47363 (local address 127.9.221.190:44917)
0811 12:03:41.293351 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:41.293359 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:41.294899 (+ 1540us) server_negotiation.cc:407] Connection header received
0811 12:03:41.296960 (+ 2061us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:41.296965 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:41.297075 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:41.297176 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:41.298709 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:41.299423 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:41.300952 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:41.301328 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:41.301416 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:41.304574 (+ 3158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:41.304579 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:41.304582 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:41.304612 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:41.307219 (+ 2607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:41.308024 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:41.308029 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:41.308031 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:41.308181 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:41.308775 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:41.308779 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:41.308781 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:41.309157 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:41.309289 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:41.310010 (+ 721us) server_negotiation.cc:299] Negotiation successful
0811 12:03:41.310173 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 12:03:41.311249 24199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:41.293789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.131:47363)
0811 12:03:41.294716 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:41.294779 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:41.296709 (+ 1930us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:41.297467 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:41.297477 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:41.297965 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:41.298543 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:41.298568 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:41.299553 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:41.299557 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:41.300776 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:41.300813 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:41.301509 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:41.301513 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:41.301697 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:41.302270 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:41.302291 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:41.304372 (+ 2081us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:03:41.307355 (+ 2983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:41.307362 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:41.307375 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:41.307911 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:41.308316 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:03:41.308323 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:03:41.308330 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:03:41.308550 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:03:41.309384 (+ 834us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:03:41.309394 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:03:41.309686 (+ 292us) client_negotiation.cc:769] Sending connection context
0811 12:03:41.309896 (+ 210us) client_negotiation.cc:240] Negotiation successful
0811 12:03:41.310240 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":638,"thread_start_us":401,"threads_started":1}
I20260811 12:03:41.312520 24195 heartbeater.cc:347] Connected to a master server at 127.9.221.190:44917
I20260811 12:03:41.312757 24195 heartbeater.cc:464] Registering TS with master...
I20260811 12:03:41.313340 24195 heartbeater.cc:511] Master 127.9.221.190:44917 requested a full tablet report, sending...
I20260811 12:03:41.314620 23528 ts_manager.cc:194] Registered new tserver with Master: bafb533e748949028cf330ffd40257d7 (127.9.221.131:43925)
I20260811 12:03:41.315388 23528 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.131@KRBTEST.COM'} at 127.9.221.131:47363
I20260811 12:03:41.321722 10102 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.9.221.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.221.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.221.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.129.keytab.
Entry for principal oryx/127.9.221.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.129.keytab.
WARNING: no policy specified for oryx/127.9.221.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.221.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.221.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.130.keytab.
Entry for principal oryx/127.9.221.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.130.keytab.
WARNING: no policy specified for oryx/127.9.221.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.221.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.221.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.131.keytab.
Entry for principal oryx/127.9.221.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.131.keytab.
I20260811 12:03:41.464229 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 23614
Aug 11 12:03:41 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.129@KRBTEST.COM
I20260811 12:03:41.480573 24208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:41.465334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34504 (local address 127.9.221.129:33255)
0811 12:03:41.465993 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:41.466002 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:41.466038 (+ 36us) server_negotiation.cc:407] Connection header received
0811 12:03:41.466140 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:41.466148 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:41.466356 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:41.466547 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:41.468019 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:41.469260 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:41.470580 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:41.471145 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:41.471278 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:41.475166 (+ 3888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:41.475177 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:41.475191 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:41.475221 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:41.477781 (+ 2560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:41.478447 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:41.478459 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:41.478461 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:41.478580 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:41.478955 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:41.478960 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:41.478963 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:41.479339 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:41.479686 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:41.479996 (+ 310us) server_negotiation.cc:299] Negotiation successful
0811 12:03:41.480251 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":290,"threads_started":1}
I20260811 12:03:41.481870 23796 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:03:41.566111 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 23614
I20260811 12:03:41.590085 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 23808
Aug 11 12:03:41 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.130@KRBTEST.COM
I20260811 12:03:41.607199 24210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:41.590965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50470 (local address 127.9.221.130:41735)
0811 12:03:41.591472 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:41.591479 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:41.591503 (+ 24us) server_negotiation.cc:407] Connection header received
0811 12:03:41.591568 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:41.591574 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:41.591828 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:41.591967 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:41.593364 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:41.594352 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:41.595919 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:41.596568 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:41.596665 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:41.601080 (+ 4415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:41.601090 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:41.601103 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:41.601135 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:41.604627 (+ 3492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:41.605195 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:41.605201 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:41.605204 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:41.605325 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:41.605629 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:41.605632 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:41.605634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:41.605971 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:41.606188 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:41.606659 (+ 471us) server_negotiation.cc:299] Negotiation successful
0811 12:03:41.606897 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":266,"threads_started":1}
I20260811 12:03:41.608510 23991 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:03:41.692938 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 23808
I20260811 12:03:41.718837 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 24016
Aug 11 12:03:41 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.131@KRBTEST.COM
I20260811 12:03:41.735466 24212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:41.719921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53814 (local address 127.9.221.131:43925)
0811 12:03:41.720510 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:41.720517 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:41.720546 (+ 29us) server_negotiation.cc:407] Connection header received
0811 12:03:41.720601 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:41.720608 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:41.720809 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:41.720971 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:41.722549 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:41.723574 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:41.725081 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:41.725652 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:41.725779 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:41.730291 (+ 4512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:41.730303 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:41.730316 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:41.730370 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:41.733049 (+ 2679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:41.733684 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:41.733690 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:41.733693 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:41.733825 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:41.734124 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:41.734127 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:41.734129 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:41.734445 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:41.734674 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:41.734866 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 12:03:41.735062 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":298,"threads_started":1}
I20260811 12:03:41.736748 24188 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:03:41.818958 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 24016
I20260811 12:03:41.842921 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 23482
I20260811 12:03:41.844205 23573 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:03:41.952751 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 23482
I20260811 12:03:41.980043 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:44917
--webserver_interface=127.9.221.190
--webserver_port=39137
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:33157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:44917
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:03:42.307808 24220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:03:42.308072 24220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:03:42.308247 24220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:03:42.315210 24220 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:03:42.315322 24220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:03:42.315415 24220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:03:42.315506 24220 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:03:42.315578 24220 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:03:42.324944 24220 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:33157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:44917
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:44917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.221.190
--webserver_port=39137
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:03:42.326169 24220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:03:42.327463 24220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:03:42.339115 24228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:42.339155 24229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:42.339327 24231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:03:42.340202 24220 server_base.cc:1034] running on GCE node
I20260811 12:03:42.340948 24220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:03:42.342278 24220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:03:42.343508 24220 hybrid_clock.cc:648] HybridClock initialized: now 1786449822343469 us; error 46 us; skew 500 ppm
I20260811 12:03:42.343922 24220 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:03:42 dist-test-slave-mjjr krb5kdc[23464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449822, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:03:42.347172 24220 init.cc:377] Logged in from keytab as kudu/127.9.221.190@KRBTEST.COM (short username kudu)
I20260811 12:03:42.348595 24220 webserver.cc:460] Webserver started at http://127.9.221.190:39137/ using document root <none> and password file <none>
I20260811 12:03:42.349232 24220 fs_manager.cc:359] Metadata directory not provided
I20260811 12:03:42.349342 24220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:03:42.354411 24220 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260811 12:03:42.357133 24239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:42.358385 24220 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 12:03:42.358521 24220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "04d4ec21ea264036a2951c2979a0d3b4"
format_stamp: "Formatted at 2026-08-11 12:03:39 on dist-test-slave-mjjr"
server_key: "ba6dd68ee5c64d5318c1f33d0ddd3208"
server_key_iv: "c5cd7e4784e78b8545958d912b67acf5"
server_key_version: "encryptionkey@0"
I20260811 12:03:42.359423 24220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:03:42.378677 24220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:03:42.382827 24220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:03:42.383049 24220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:03:42.401436 24220 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.190:44917
I20260811 12:03:42.401481 24309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.190:44917 every 8 connection(s)
I20260811 12:03:42.402949 24220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:03:42.409637 24310 sys_catalog.cc:263] Verifying existing consensus state
I20260811 12:03:42.412914 24310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04d4ec21ea264036a2951c2979a0d3b4: Bootstrap starting.
I20260811 12:03:42.413352 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 24220
I20260811 12:03:42.414356 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.129:33255
--local_ip_for_outbound_sockets=127.9.221.129
--tserver_master_addrs=127.9.221.190:44917
--webserver_port=43379
--webserver_interface=127.9.221.129
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab
--principal=kudu/127.9.221.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:33157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.221.129
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:03:42.435601 24310 log.cc:824] T 00000000000000000000000000000000 P 04d4ec21ea264036a2951c2979a0d3b4: Log is configured to *not* fsync() on all Append() calls
I20260811 12:03:42.444379 24310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04d4ec21ea264036a2951c2979a0d3b4: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 12:03:42.444895 24310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04d4ec21ea264036a2951c2979a0d3b4: Bootstrap complete.
I20260811 12:03:42.454001 24310 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04d4ec21ea264036a2951c2979a0d3b4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04d4ec21ea264036a2951c2979a0d3b4" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 44917 } }
I20260811 12:03:42.454835 24310 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04d4ec21ea264036a2951c2979a0d3b4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04d4ec21ea264036a2951c2979a0d3b4, State: Initialized, Role: FOLLOWER
I20260811 12:03:42.455444 24310 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04d4ec21ea264036a2951c2979a0d3b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04d4ec21ea264036a2951c2979a0d3b4" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 44917 } }
I20260811 12:03:42.455696 24310 raft_consensus.cc:388] T 00000000000000000000000000000000 P 04d4ec21ea264036a2951c2979a0d3b4 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:03:42.455837 24310 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04d4ec21ea264036a2951c2979a0d3b4 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:03:42.456022 24310 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04d4ec21ea264036a2951c2979a0d3b4 [term 1 FOLLOWER]: Advancing to term 2
I20260811 12:03:42.459854 24310 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04d4ec21ea264036a2951c2979a0d3b4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04d4ec21ea264036a2951c2979a0d3b4" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 44917 } }
I20260811 12:03:42.460359 24310 leader_election.cc:302] T 00000000000000000000000000000000 P 04d4ec21ea264036a2951c2979a0d3b4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04d4ec21ea264036a2951c2979a0d3b4; no voters:
I20260811 12:03:42.461005 24310 leader_election.cc:288] T 00000000000000000000000000000000 P 04d4ec21ea264036a2951c2979a0d3b4 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 12:03:42.461144 24316 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04d4ec21ea264036a2951c2979a0d3b4 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 12:03:42.462373 24316 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04d4ec21ea264036a2951c2979a0d3b4 [term 2 LEADER]: Becoming Leader. State: Replica: 04d4ec21ea264036a2951c2979a0d3b4, State: Running, Role: LEADER
I20260811 12:03:42.463014 24316 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04d4ec21ea264036a2951c2979a0d3b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04d4ec21ea264036a2951c2979a0d3b4" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 44917 } }
I20260811 12:03:42.463310 24310 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04d4ec21ea264036a2951c2979a0d3b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:03:42.465551 24320 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04d4ec21ea264036a2951c2979a0d3b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04d4ec21ea264036a2951c2979a0d3b4. Latest consensus state: current_term: 2 leader_uuid: "04d4ec21ea264036a2951c2979a0d3b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04d4ec21ea264036a2951c2979a0d3b4" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 44917 } } }
I20260811 12:03:42.465976 24320 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04d4ec21ea264036a2951c2979a0d3b4 [sys.catalog]: This master's current role is: LEADER
I20260811 12:03:42.466210 24318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04d4ec21ea264036a2951c2979a0d3b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "04d4ec21ea264036a2951c2979a0d3b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04d4ec21ea264036a2951c2979a0d3b4" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 44917 } } }
I20260811 12:03:42.466828 24318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04d4ec21ea264036a2951c2979a0d3b4 [sys.catalog]: This master's current role is: LEADER
I20260811 12:03:42.467613 24327 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:03:42.479235 24327 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:03:42.480676 24327 catalog_manager.cc:1259] Loaded cluster ID: 18026015fd3b4cdd96912bbad3d82074
I20260811 12:03:42.480824 24327 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:03:42.484756 24327 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:03:42.493397 24327 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 04d4ec21ea264036a2951c2979a0d3b4: Loaded TSK: 0
I20260811 12:03:42.494930 24327 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 12:03:42.763573 24314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:03:42.763935 24314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:03:42.764110 24314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:03:42.771274 24314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:03:42.771409 24314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:03:42.771601 24314 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.129
I20260811 12:03:42.781154 24314 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:33157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.129.keytab
--principal=oryx/127.9.221.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.129:33255
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.221.129
--webserver_port=43379
--tserver_master_addrs=127.9.221.190:44917
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.129
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:03:42.782419 24314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:03:42.783725 24314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:03:42.798398 24348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:42.798372 24349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:03:42.799538 24314 server_base.cc:1034] running on GCE node
W20260811 12:03:42.799683 24356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:03:42.800298 24314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:03:42.801288 24314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:03:42.802521 24314 hybrid_clock.cc:648] HybridClock initialized: now 1786449822802466 us; error 48 us; skew 500 ppm
I20260811 12:03:42.802852 24314 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:03:42 dist-test-slave-mjjr krb5kdc[23464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449822, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.221.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:03:42.811189 24314 init.cc:377] Logged in from keytab as oryx/127.9.221.129@KRBTEST.COM (short username oryx)
I20260811 12:03:42.812774 24314 webserver.cc:460] Webserver started at http://127.9.221.129:43379/ using document root <none> and password file <none>
I20260811 12:03:42.813436 24314 fs_manager.cc:359] Metadata directory not provided
I20260811 12:03:42.813576 24314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:03:42.818930 24314 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 12:03:42.821684 24364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:42.823063 24314 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:03:42.823227 24314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-0/wal
uuid: "9b199daf7b5d4f1ebab0647ca447aad2"
format_stamp: "Formatted at 2026-08-11 12:03:40 on dist-test-slave-mjjr"
server_key: "dc480eb3c875339ec3221b2de5a13494"
server_key_iv: "769355e45cfcff6d9cf87a8584383092"
server_key_version: "encryptionkey@0"
I20260811 12:03:42.824242 24314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:03:42.849050 24314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:03:42.853287 24314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:03:42.853560 24314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:03:42.854516 24314 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:03:42.855872 24314 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:03:42.855965 24314 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:42.856113 24314 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:03:42.856205 24314 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:42.889889 24314 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.129:33255
I20260811 12:03:42.889983 24523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.129:33255 every 8 connection(s)
I20260811 12:03:42.891316 24314 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
I20260811 12:03:42.893282 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 24314
I20260811 12:03:42.893961 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.130:41735
--local_ip_for_outbound_sockets=127.9.221.130
--tserver_master_addrs=127.9.221.190:44917
--webserver_port=41737
--webserver_interface=127.9.221.130
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab
--principal=kudu/127.9.221.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:33157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.221.130
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 12:03:42 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.129@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:42 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.129@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 12:03:43.245870 24530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:03:43.246150 24530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:03:43.246313 24530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:03:43.253499 24530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:03:43.253629 24530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:03:43.253824 24530 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.130
I20260811 12:03:43.263427 24530 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:33157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.130.keytab
--principal=oryx/127.9.221.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.130:41735
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.221.130
--webserver_port=41737
--tserver_master_addrs=127.9.221.190:44917
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.130
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:03:43.264789 24530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:03:43.266139 24530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:03:43.285122 24544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:43.285161 24542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:43.285912 24541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:03:43.286569 24530 server_base.cc:1034] running on GCE node
I20260811 12:03:43.287711 24530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:03:43.289572 24530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:03:43.290891 24530 hybrid_clock.cc:648] HybridClock initialized: now 1786449823290810 us; error 47 us; skew 500 ppm
I20260811 12:03:43.291432 24530 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:03:43 dist-test-slave-mjjr krb5kdc[23464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449823, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.221.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:03:43.296391 24530 init.cc:377] Logged in from keytab as oryx/127.9.221.130@KRBTEST.COM (short username oryx)
I20260811 12:03:43.298079 24530 webserver.cc:460] Webserver started at http://127.9.221.130:41737/ using document root <none> and password file <none>
I20260811 12:03:43.298758 24530 fs_manager.cc:359] Metadata directory not provided
I20260811 12:03:43.298915 24530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:03:43.304754 24530 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 12:03:43.307792 24554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:43.309440 24530 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 12:03:43.309631 24530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-1/wal
uuid: "747455b70d10478fb9d430e4638c59ab"
format_stamp: "Formatted at 2026-08-11 12:03:40 on dist-test-slave-mjjr"
server_key: "500c4996b253020f1baaa677dbeceb61"
server_key_iv: "fbd5e290352294ce1a614981b138a677"
server_key_version: "encryptionkey@0"
I20260811 12:03:43.310587 24530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:03:43.339733 24530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:03:43.344017 24530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:03:43.344327 24530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:03:43.345361 24530 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:03:43.346812 24530 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:03:43.346936 24530 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:43.347066 24530 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:03:43.347153 24530 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:43.395780 24530 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.130:41735
I20260811 12:03:43.395931 24740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.130:41735 every 8 connection(s)
I20260811 12:03:43.397236 24530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
I20260811 12:03:43.404299 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 24530
I20260811 12:03:43.405287 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.131:43925
--local_ip_for_outbound_sockets=127.9.221.131
--tserver_master_addrs=127.9.221.190:44917
--webserver_port=41007
--webserver_interface=127.9.221.131
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab
--principal=kudu/127.9.221.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:33157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.221.131
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 12:03:43 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.130@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:43 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.130@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 12:03:43.781576 24748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:03:43.781926 24748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:03:43.782104 24748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:03:43.789705 24748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:03:43.789873 24748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:03:43.790068 24748 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.131
I20260811 12:03:43.800100 24748 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:33157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/krb5kdc/oryx_127.9.221.131.keytab
--principal=oryx/127.9.221.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.131:43925
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.221.131
--webserver_port=41007
--tserver_master_addrs=127.9.221.190:44917
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.131
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:03:43.801469 24748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:03:43.802822 24748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:03:43.821406 24760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:43.832791 24761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:43.834242 24764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:03:43.834713 24748 server_base.cc:1034] running on GCE node
I20260811 12:03:43.835973 24748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:03:43.838001 24748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:03:43.839416 24748 hybrid_clock.cc:648] HybridClock initialized: now 1786449823839308 us; error 75 us; skew 500 ppm
I20260811 12:03:43.840072 24748 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:03:43 dist-test-slave-mjjr krb5kdc[23464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449823, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.221.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:03:43.846836 24748 init.cc:377] Logged in from keytab as oryx/127.9.221.131@KRBTEST.COM (short username oryx)
I20260811 12:03:43.849654 24748 webserver.cc:460] Webserver started at http://127.9.221.131:41007/ using document root <none> and password file <none>
I20260811 12:03:43.851012 24748 fs_manager.cc:359] Metadata directory not provided
I20260811 12:03:43.851233 24748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:03:43.861637 24748 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.006s sys 0.002s
I20260811 12:03:43.876190 24774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:43.879136 24748 fs_manager.cc:725] Time spent opening block manager: real 0.015s user 0.001s sys 0.009s
I20260811 12:03:43.879483 24748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-2/wal
uuid: "bafb533e748949028cf330ffd40257d7"
format_stamp: "Formatted at 2026-08-11 12:03:41 on dist-test-slave-mjjr"
server_key: "9e46f57954cc07370be0affe3e92879c"
server_key_iv: "76c8193359608690559d688b9825aab4"
server_key_version: "encryptionkey@0"
I20260811 12:03:43.881523 24748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:03:43.922457 24748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:03:43.926769 24748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:03:43.927052 24748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:03:43.928411 24748 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:03:43.929919 24748 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:03:43.930037 24748 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:43.930166 24748 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:03:43.930264 24748 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:43.969909 24748 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.131:43925
I20260811 12:03:43.969969 24934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.131:43925 every 8 connection(s)
I20260811 12:03:43.971401 24748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
I20260811 12:03:43.976943 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 24748
Aug 11 12:03:44 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.131@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:44 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.131@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:03:44.055044 24943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:43.978745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45946 (local address 127.9.221.190:44917)
0811 12:03:44.015093 (+ 36348us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:44.015104 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:44.020712 (+ 5608us) server_negotiation.cc:407] Connection header received
0811 12:03:44.021173 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:44.021184 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:44.021335 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:44.021471 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:44.025357 (+ 3886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:44.027919 (+ 2562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:44.031191 (+ 3272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:44.032295 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:44.032456 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:44.039322 (+ 6866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:44.039341 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:44.039378 (+ 37us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:44.039460 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:44.046037 (+ 6577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:44.047541 (+ 1504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:44.047559 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:44.047575 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:44.047933 (+ 358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:03:44.048704 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:03:44.048712 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:03:44.048716 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:03:44.049850 (+ 1134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:44.050324 (+ 474us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:44.052508 (+ 2184us) server_negotiation.cc:299] Negotiation successful
0811 12:03:44.053284 (+ 776us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":36110,"thread_start_us":32389,"threads_started":1}
W20260811 12:03:44.356005 24529 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.449s user 0.004s sys 0.012s
W20260811 12:03:44.358270 24528 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:42.894445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:51211 (local address 127.9.221.190:44917)
0811 12:03:42.895244 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:42.895273 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:42.896443 (+ 1170us) server_negotiation.cc:407] Connection header received
0811 12:03:42.899517 (+ 3074us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:42.899529 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:42.899904 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:42.900222 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:42.902312 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:42.903562 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:42.905167 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:42.905807 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:42.906013 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:44.357769 (+1451756us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.129:51211: BlockingRecv error: recv got EOF from 127.9.221.129:51211 (error 108)
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":272,"threads_started":1}
I20260811 12:03:44.359455 24529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:42.894473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.129:51211)
0811 12:03:42.896235 (+ 1762us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:42.896295 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:42.898961 (+ 2666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:42.900457 (+ 1496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:42.900470 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:42.901272 (+ 802us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:42.902035 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:42.902062 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:42.903765 (+ 1703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:42.903773 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:42.905010 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:42.905021 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:42.906054 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:42.906061 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:42.906328 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:42.907012 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:42.907033 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:44.357306 (+1450273us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1463,"thread_start_us":1244,"threads_started":1}
W20260811 12:03:44.361526 24526 heartbeater.cc:644] Failed to heartbeat to 127.9.221.190:44917 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.221.190:44917: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Aug 11 12:03:44 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.129@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:44 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.129@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:03:44.379498 24529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:44.362577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.129:49185)
0811 12:03:44.363456 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:44.363486 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:44.363690 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:44.364534 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:44.364542 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:44.365203 (+ 661us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:44.365514 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:44.365531 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:44.367329 (+ 1798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:44.367337 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:44.369630 (+ 2293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:44.369651 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:44.371075 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:44.371084 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:44.371408 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:44.372571 (+ 1163us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:44.372607 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:44.379017 (+ 6410us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":669,"mutex_wait_us":572}
W20260811 12:03:44.379765 24528 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:44.362848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:49185 (local address 127.9.221.190:44917)
0811 12:03:44.363111 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:44.363120 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:44.363596 (+ 476us) server_negotiation.cc:407] Connection header received
0811 12:03:44.363925 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:44.363934 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:44.364090 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:44.364190 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:44.365695 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:44.367126 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:44.369825 (+ 2699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:44.370820 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:44.371001 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:44.379364 (+ 8363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.129:49185: BlockingRecv error: recv got EOF from 127.9.221.129:49185 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Aug 11 12:03:44 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.129@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:44 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.129@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:03:44.397078 24529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:44.381306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.129:33399)
0811 12:03:44.381844 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:44.381872 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:44.382031 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:44.382554 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:44.382562 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:44.383132 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:44.383449 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:44.383464 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:44.385569 (+ 2105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:44.385579 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:44.387848 (+ 2269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:44.387870 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:44.389615 (+ 1745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:44.389624 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:44.390000 (+ 376us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:44.390964 (+ 964us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:44.390998 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:44.396679 (+ 5681us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":383}
W20260811 12:03:44.398151 24528 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:44.381436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:33399 (local address 127.9.221.190:44917)
0811 12:03:44.381657 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:44.381666 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:44.381976 (+ 310us) server_negotiation.cc:407] Connection header received
0811 12:03:44.382180 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:44.382188 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:44.382332 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:44.382473 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:44.383632 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:44.385299 (+ 1667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:44.388064 (+ 2765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:44.389395 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:44.389590 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:44.397788 (+ 8198us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.129:33399: BlockingRecv error: recv got EOF from 127.9.221.129:33399 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260811 12:03:44.398986 24526 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 12:03:44.818380 24747 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.387s user 0.008s sys 0.011s
W20260811 12:03:44.819772 24746 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:43.401079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:59523 (local address 127.9.221.190:44917)
0811 12:03:43.416613 (+ 15534us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:43.416624 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:43.419728 (+ 3104us) server_negotiation.cc:407] Connection header received
0811 12:03:43.421771 (+ 2043us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:43.421776 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:43.421880 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:43.422018 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:43.424373 (+ 2355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:43.425662 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:43.428489 (+ 2827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:43.429278 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:43.429422 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:44.819429 (+1390007us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.130:59523: BlockingRecv error: recv got EOF from 127.9.221.130:59523 (error 108)
Metrics: {"server-negotiator.queue_time_us":15284,"thread_start_us":18613,"threads_started":1}
I20260811 12:03:44.820413 24747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:43.401586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.130:59523)
0811 12:03:43.417515 (+ 15929us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:43.417589 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:43.421319 (+ 3730us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:43.422506 (+ 1187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:43.422518 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:43.423268 (+ 750us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:43.424114 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:43.424140 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:43.425889 (+ 1749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:43.425896 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:43.428297 (+ 2401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:43.428316 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:43.429803 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:43.429813 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:43.430167 (+ 354us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:43.431009 (+ 842us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:43.431044 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:44.819217 (+1388173us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":15435,"thread_start_us":15162,"threads_started":1}
W20260811 12:03:44.821789 24744 heartbeater.cc:644] Failed to heartbeat to 127.9.221.190:44917 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.221.190:44917: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Aug 11 12:03:44 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.130@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:44 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.130@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:03:44.832216 24747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:44.822767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.130:60363)
0811 12:03:44.823083 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:44.823099 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:44.823262 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:44.823589 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:44.823594 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:44.823995 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:44.824160 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:44.824168 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:44.825218 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:44.825224 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:44.826141 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:44.826152 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:44.826914 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:44.826920 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:44.827091 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:44.827762 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:44.827866 (+ 104us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:44.831928 (+ 4062us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212}
W20260811 12:03:44.832387 24746 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:44.822793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:60363 (local address 127.9.221.190:44917)
0811 12:03:44.822958 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:44.822962 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:44.823188 (+ 226us) server_negotiation.cc:407] Connection header received
0811 12:03:44.823388 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:44.823393 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:44.823464 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:44.823561 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:44.824304 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:44.825078 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:44.826294 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:44.826741 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:44.826881 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:44.832122 (+ 5241us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.130:60363: BlockingRecv error: recv got EOF from 127.9.221.130:60363 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 11 12:03:44 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.130@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:44 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.130@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:03:44.844892 24747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:44.833030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.130:54267)
0811 12:03:44.833284 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:44.833316 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:44.833487 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:44.833969 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:44.833978 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:44.834630 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:44.834953 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:44.834972 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:44.836070 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:44.836081 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:44.838598 (+ 2517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:44.838621 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:44.839536 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:44.839547 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:44.839935 (+ 388us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:44.840762 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:44.840785 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:44.844718 (+ 3933us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 12:03:44.845322 24744 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 12:03:44.845567 24746 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:44.833189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:54267 (local address 127.9.221.190:44917)
0811 12:03:44.833327 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:44.833331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:44.833391 (+ 60us) server_negotiation.cc:407] Connection header received
0811 12:03:44.833731 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:44.833734 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:44.833790 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:44.833874 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:44.835137 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:44.835891 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:44.838947 (+ 3056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:44.839340 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:44.839439 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:44.845112 (+ 5673us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.130:54267: BlockingRecv error: recv got EOF from 127.9.221.130:54267 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260811 12:03:45.233675 24942 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.199s user 0.006s sys 0.020s
W20260811 12:03:45.234915 24944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:43.976909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:47103 (local address 127.9.221.190:44917)
0811 12:03:44.011101 (+ 34192us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:44.011112 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:44.019727 (+ 8615us) server_negotiation.cc:407] Connection header received
0811 12:03:44.021835 (+ 2108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:44.021845 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:44.021961 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:44.022104 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:44.025779 (+ 3675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:44.028087 (+ 2308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:44.031790 (+ 3703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:44.032810 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:44.032974 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:45.234694 (+1201720us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.131:47103: BlockingRecv error: recv got EOF from 127.9.221.131:47103 (error 108)
Metrics: {"mutex_wait_us":153,"server-negotiator.queue_time_us":33965,"thread_start_us":37755,"threads_started":1}
I20260811 12:03:45.235447 24942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:43.976104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.131:47103)
0811 12:03:44.016093 (+ 39989us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:44.016188 (+ 95us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:44.021065 (+ 4877us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:44.022848 (+ 1783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:44.022868 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:44.024128 (+ 1260us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:44.025354 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:44.025406 (+ 52us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:44.028376 (+ 2970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:44.028385 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:44.031503 (+ 3118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:44.031527 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:44.033487 (+ 1960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:44.033497 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:44.033964 (+ 467us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:44.034963 (+ 999us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:44.035011 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:45.234459 (+1199448us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":39435,"thread_start_us":39111,"threads_started":1}
W20260811 12:03:45.236469 24935 heartbeater.cc:644] Failed to heartbeat to 127.9.221.190:44917 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.221.190:44917: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Aug 11 12:03:45 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.131@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:45 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.131@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:03:45.246002 24942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:45.237138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.131:56005)
0811 12:03:45.237402 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:45.237425 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:45.237546 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:45.237900 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:45.237906 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:45.238280 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:45.238441 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:45.238449 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:45.239478 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:45.239487 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:45.240652 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:45.240669 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:45.241674 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:45.241680 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:45.241880 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:45.242436 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:45.242453 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:45.245804 (+ 3351us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":87}
W20260811 12:03:45.246191 24944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:45.237242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:56005 (local address 127.9.221.190:44917)
0811 12:03:45.237439 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:45.237447 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:45.237495 (+ 48us) server_negotiation.cc:407] Connection header received
0811 12:03:45.237680 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:45.237685 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:45.237752 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:45.237853 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:45.238577 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:45.239337 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:45.240783 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:45.241468 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:45.241588 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:45.245992 (+ 4404us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.131:56005: BlockingRecv error: recv got EOF from 127.9.221.131:56005 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 11 12:03:45 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.131@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:45 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.131@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:03:45.254714 24942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:45.246651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.131:55941)
0811 12:03:45.246800 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:45.246823 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:45.246937 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:45.247282 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:45.247288 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:45.247613 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:45.247814 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:45.247822 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:45.248781 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:45.248786 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:45.249741 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:45.249753 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:45.250490 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:45.250496 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:45.250680 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:45.251232 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:45.251249 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:45.254530 (+ 3281us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 12:03:45.254957 24944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:45.246746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:55941 (local address 127.9.221.190:44917)
0811 12:03:45.247020 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:45.247027 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:45.247042 (+ 15us) server_negotiation.cc:407] Connection header received
0811 12:03:45.247084 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:45.247087 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:45.247150 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:45.247249 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:45.247947 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:45.248641 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:45.249884 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:45.250332 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:45.250465 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:45.254708 (+ 4243us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.131:55941: BlockingRecv error: recv got EOF from 127.9.221.131:55941 (error 108)
Metrics: {"server-negotiator.queue_time_us":137}
W20260811 12:03:45.255041 24935 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 11 12:03:45 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.129@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:45 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.129@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:03:45.408993 24947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:45.399501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.129:52841)
0811 12:03:45.400008 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:45.400023 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:45.400169 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:45.400498 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:45.400503 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:45.400987 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:45.401149 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:45.401159 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:45.402149 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:45.402156 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:45.403104 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:45.403115 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:45.403952 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:45.403959 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:45.404126 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:45.404745 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:45.404767 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:45.408666 (+ 3899us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":290,"threads_started":1}
W20260811 12:03:45.409066 24944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:45.399589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:52841 (local address 127.9.221.190:44917)
0811 12:03:45.399814 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:45.399822 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:45.400103 (+ 281us) server_negotiation.cc:407] Connection header received
0811 12:03:45.400292 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:45.400296 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:45.400359 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:45.400474 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:45.401298 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:45.402003 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:45.403245 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:45.403757 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:45.403908 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:45.408853 (+ 4945us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.129:52841: BlockingRecv error: recv got EOF from 127.9.221.129:52841 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 11 12:03:45 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.130@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:45 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.130@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:03:45.854846 24948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:45.846065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.130:35823)
0811 12:03:45.846585 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:45.846600 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:45.846715 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:45.847004 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:45.847008 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:45.847430 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:45.847575 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:45.847583 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:45.848586 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:45.848589 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:45.849546 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:45.849557 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:45.850210 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:45.850214 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:45.850327 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:45.850860 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:45.850876 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:45.854492 (+ 3616us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":296,"threads_started":1}
W20260811 12:03:45.854969 24944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:45.846106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:35823 (local address 127.9.221.190:44917)
0811 12:03:45.846321 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:45.846330 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:45.846675 (+ 345us) server_negotiation.cc:407] Connection header received
0811 12:03:45.846810 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:45.846814 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:45.846874 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:45.846955 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:45.847740 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:45.848468 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:45.849662 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:45.850063 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:45.850156 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:45.854721 (+ 4565us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.130:35823: BlockingRecv error: recv got EOF from 127.9.221.130:35823 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 11 12:03:46 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.131@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:46 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.131@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:03:46.265038 24949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:46.255625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.131:42345)
0811 12:03:46.256194 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:46.256209 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:46.256358 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:46.256716 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:46.256720 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:46.257185 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:46.257331 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:46.257340 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:46.258356 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:46.258363 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:46.259348 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:46.259359 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:46.260182 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:46.260189 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:46.260358 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:46.260969 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:46.260986 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:46.264684 (+ 3698us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":315,"threads_started":1}
W20260811 12:03:46.265209 24944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:46.255737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:42345 (local address 127.9.221.190:44917)
0811 12:03:46.255963 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:46.255971 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:46.256293 (+ 322us) server_negotiation.cc:407] Connection header received
0811 12:03:46.256491 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:46.256496 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:46.256580 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:46.256680 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:46.257487 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:46.258222 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:46.259495 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:46.259998 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:46.260123 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:46.264915 (+ 4792us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.131:42345: BlockingRecv error: recv got EOF from 127.9.221.131:42345 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 11 12:03:46 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.129@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:46 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.129@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:03:46.419585 24950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:46.410017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.129:46673)
0811 12:03:46.410522 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:46.410538 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:46.410680 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:46.411010 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:46.411015 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:46.411498 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:46.411723 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:46.411736 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:46.412743 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:46.412749 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:46.413712 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:46.413749 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:46.414457 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:46.414463 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:46.414646 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:46.415240 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:46.415259 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:46.419191 (+ 3932us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":270,"threads_started":1}
W20260811 12:03:46.419766 24944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:46.410058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:46673 (local address 127.9.221.190:44917)
0811 12:03:46.410258 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:46.410267 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:46.410615 (+ 348us) server_negotiation.cc:407] Connection header received
0811 12:03:46.410813 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:46.410818 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:46.410877 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:46.410985 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:46.411870 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:46.412611 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:46.413864 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:46.414289 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:46.414415 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:46.419422 (+ 5007us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.129:46673: BlockingRecv error: recv got EOF from 127.9.221.129:46673 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 11 12:03:46 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.130@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:46 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.130@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:03:46.864578 24951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:46.855817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.130:46775)
0811 12:03:46.856407 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:46.856423 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:46.856548 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:46.856836 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:46.856840 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:46.857232 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:46.857416 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:46.857424 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:46.858368 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:46.858372 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:46.859303 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:46.859313 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:46.859960 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:46.859965 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:46.860086 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:46.860672 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:46.860690 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:46.864250 (+ 3560us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":347,"threads_started":1}
W20260811 12:03:46.865449 24944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:46.855849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:46775 (local address 127.9.221.190:44917)
0811 12:03:46.856021 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:46.856025 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:46.856502 (+ 477us) server_negotiation.cc:407] Connection header received
0811 12:03:46.856646 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:46.856650 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:46.856710 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:46.856790 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:46.857549 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:46.858255 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:46.859416 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:46.859839 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:46.859929 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:46.865195 (+ 5266us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.130:46775: BlockingRecv error: recv got EOF from 127.9.221.130:46775 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 11 12:03:47 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.131@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:47 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.131@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:03:47.280131 24953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:47.266190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.131:35773)
0811 12:03:47.266753 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:47.266770 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:47.266924 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:47.267280 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:47.267285 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:47.267816 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:47.268001 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:47.268010 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:47.269052 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:47.269059 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:47.270282 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:47.270303 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:47.271003 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:47.271009 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:47.271191 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:47.276024 (+ 4833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:47.276042 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:47.279870 (+ 3828us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":337,"threads_started":1}
W20260811 12:03:47.280319 24944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:47.266281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:35773 (local address 127.9.221.190:44917)
0811 12:03:47.266511 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:47.266519 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:47.266858 (+ 339us) server_negotiation.cc:407] Connection header received
0811 12:03:47.267052 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:47.267058 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:47.267141 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:47.267243 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:47.268147 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:47.268892 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:47.270430 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:47.270839 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:47.270981 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:47.280085 (+ 9104us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.131:35773: BlockingRecv error: recv got EOF from 127.9.221.131:35773 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 11 12:03:47 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.129@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:47 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.129@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:03:47.429930 24954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:47.420547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.129:46709)
0811 12:03:47.421074 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:47.421090 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:47.421240 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:47.421598 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:47.421603 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:47.421992 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:47.422164 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:47.422172 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:47.423212 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:47.423216 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:47.424412 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:47.424434 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:47.425359 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:47.425366 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:47.425543 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:47.426086 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:47.426101 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:47.429575 (+ 3474us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":268,"threads_started":1}
W20260811 12:03:47.430076 24944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:47.420635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:46709 (local address 127.9.221.190:44917)
0811 12:03:47.420864 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:47.420885 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:47.421169 (+ 284us) server_negotiation.cc:407] Connection header received
0811 12:03:47.421394 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:47.421398 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:47.421461 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:47.421564 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:47.422308 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:47.423063 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:47.424564 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:47.425160 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:47.425296 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:47.429791 (+ 4495us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.129:46709: BlockingRecv error: recv got EOF from 127.9.221.129:46709 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 11 12:03:47 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.130@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:47 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.130@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:03:47.881093 24955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:47.866574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.130:41791)
0811 12:03:47.868520 (+ 1946us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:47.868540 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:47.868691 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:47.869060 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:47.869066 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:47.869547 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:47.869779 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:47.869794 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:47.871382 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:47.871389 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:47.872804 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:47.872831 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:47.873732 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:47.873739 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:47.873949 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:47.877487 (+ 3538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:47.877507 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:47.880828 (+ 3321us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1810,"thread_start_us":1670,"threads_started":1}
W20260811 12:03:47.881227 24944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:47.866653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:41791 (local address 127.9.221.190:44917)
0811 12:03:47.866901 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:47.866910 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:47.868630 (+ 1720us) server_negotiation.cc:407] Connection header received
0811 12:03:47.868845 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:47.868851 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:47.868923 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:47.869026 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:47.870023 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:47.871212 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:47.872988 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:47.873561 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:47.873702 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:47.881018 (+ 7316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.130:41791: BlockingRecv error: recv got EOF from 127.9.221.130:41791 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 11 12:03:48 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.131@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:48 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.131@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:03:48.290045 24956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:48.281198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.131:42505)
0811 12:03:48.281658 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:48.281674 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:48.281788 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:48.282101 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:48.282104 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:48.282465 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:48.282624 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:48.282632 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:48.283595 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:48.283598 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:48.284527 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:48.284537 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:48.285153 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:48.285157 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:48.285278 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:48.285827 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:48.285843 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:48.289720 (+ 3877us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":246,"threads_started":1}
W20260811 12:03:48.290092 24944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:48.281294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:42505 (local address 127.9.221.190:44917)
0811 12:03:48.281547 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:48.281553 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:48.281754 (+ 201us) server_negotiation.cc:407] Connection header received
0811 12:03:48.281886 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:48.281889 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:48.281977 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:48.282060 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:48.282750 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:48.283474 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:48.284636 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:48.285038 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:48.285136 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:48.289915 (+ 4779us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.131:42505: BlockingRecv error: recv got EOF from 127.9.221.131:42505 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
Aug 11 12:03:48 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.129@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:48 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.129@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:03:48.439512 24958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:48.430956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.129:34209)
0811 12:03:48.431406 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:48.431423 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:48.431551 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:48.431852 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:48.431856 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:48.432253 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:48.432419 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:48.432427 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:48.433408 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:48.433411 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:48.434384 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:48.434396 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:48.434994 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:48.434997 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:48.435118 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:48.435774 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:48.435791 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:48.439237 (+ 3446us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":250,"threads_started":1}
W20260811 12:03:48.439682 24944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:48.431028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:34209 (local address 127.9.221.190:44917)
0811 12:03:48.431255 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:48.431260 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:48.431519 (+ 259us) server_negotiation.cc:407] Connection header received
0811 12:03:48.431658 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:48.431661 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:48.431722 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:48.431800 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:48.432552 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:48.433298 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:48.434504 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:48.434880 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:48.434969 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:48.439422 (+ 4453us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.129:34209: BlockingRecv error: recv got EOF from 127.9.221.129:34209 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 11 12:03:48 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.130@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:48 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.130@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:03:48.892073 24960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:48.882359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.130:43561)
0811 12:03:48.882957 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:48.882972 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:48.883121 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:48.883552 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:48.883558 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:48.884068 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:48.884232 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:48.884241 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:48.885215 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:48.885219 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:48.886195 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:48.886206 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:48.886920 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:48.886924 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:48.887057 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:48.887847 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:48.887865 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:48.891721 (+ 3856us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":359,"threads_started":1}
W20260811 12:03:48.892194 24944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:48.882492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:43561 (local address 127.9.221.190:44917)
0811 12:03:48.882741 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:48.882750 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:48.883077 (+ 327us) server_negotiation.cc:407] Connection header received
0811 12:03:48.883282 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:48.883289 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:48.883374 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:48.883484 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:48.884363 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:48.885088 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:48.886315 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:48.886754 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:48.886846 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:48.891926 (+ 5080us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.130:43561: BlockingRecv error: recv got EOF from 127.9.221.130:43561 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 11 12:03:49 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.131@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:49 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.131@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:03:49.300310 24961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:49.291016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.131:57217)
0811 12:03:49.291554 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:49.291570 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:49.291748 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:49.292127 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:49.292132 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:49.292546 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:49.292707 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:49.292715 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:49.293754 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:49.293760 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:49.294766 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:49.294777 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:49.295474 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:49.295480 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:49.295691 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:49.296235 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:49.296253 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:49.299980 (+ 3727us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":278,"threads_started":1}
W20260811 12:03:49.300554 24944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:49.291085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:57217 (local address 127.9.221.190:44917)
0811 12:03:49.291314 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:49.291322 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:49.291667 (+ 345us) server_negotiation.cc:407] Connection header received
0811 12:03:49.291875 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:49.291881 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:49.291981 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:49.292088 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:49.292853 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:49.293614 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:49.294909 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:49.295331 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:49.295474 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:49.300244 (+ 4770us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.131:57217: BlockingRecv error: recv got EOF from 127.9.221.131:57217 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 11 12:03:49 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.129@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:49 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.129@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:03:49.449445 24962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:49.440677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.129:60947)
0811 12:03:49.441201 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:49.441222 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:49.441389 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:49.441780 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:49.441787 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:49.442362 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:49.442521 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:49.442531 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:49.443456 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:49.443459 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:49.444478 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:49.444490 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:49.445107 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:49.445112 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:49.445236 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:49.445861 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:49.445879 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:49.449225 (+ 3346us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":251,"threads_started":1}
W20260811 12:03:49.449637 24944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:49.440738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:60947 (local address 127.9.221.190:44917)
0811 12:03:49.440926 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:49.440932 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:49.441315 (+ 383us) server_negotiation.cc:407] Connection header received
0811 12:03:49.441541 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:49.441549 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:49.441635 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:49.441716 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:49.442654 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:49.443347 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:49.444602 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:49.444987 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:49.445079 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:49.449396 (+ 4317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.129:60947: BlockingRecv error: recv got EOF from 127.9.221.129:60947 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 11 12:03:49 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.130@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:49 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.130@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:03:49.901885 24963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:49.893191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.130:34509)
0811 12:03:49.893808 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:49.893824 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:49.893972 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:49.894398 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:49.894403 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:49.894923 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:49.895100 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:49.895109 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:49.896149 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:49.896152 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:49.897038 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:49.897048 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:49.897798 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:49.897802 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:49.897923 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:49.898466 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:49.898483 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:49.901679 (+ 3196us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":348,"threads_started":1}
W20260811 12:03:49.902108 24944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:49.893432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:34509 (local address 127.9.221.190:44917)
0811 12:03:49.893648 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:49.893657 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:49.893908 (+ 251us) server_negotiation.cc:407] Connection header received
0811 12:03:49.894126 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:49.894132 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:49.894232 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:49.894345 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:49.895246 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:49.896031 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:49.897173 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:49.897679 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:49.897773 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:49.901865 (+ 4092us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.130:34509: BlockingRecv error: recv got EOF from 127.9.221.130:34509 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 11 12:03:50 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.131@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:50 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.131@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:03:50.310267 24966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:50.301434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.131:33931)
0811 12:03:50.301896 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:50.301913 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:50.302028 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:50.302327 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:50.302330 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:50.302767 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:50.302910 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:50.302919 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:50.303922 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:50.303925 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:50.304862 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:50.304873 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:50.305506 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:50.305509 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:50.305638 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:50.306224 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:50.306240 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:50.310035 (+ 3795us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":241,"threads_started":1}
W20260811 12:03:50.310447 24944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:50.301494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:33931 (local address 127.9.221.190:44917)
0811 12:03:50.301730 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:50.301737 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:50.301996 (+ 259us) server_negotiation.cc:407] Connection header received
0811 12:03:50.302126 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:50.302130 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:50.302206 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:50.302286 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:50.303034 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:50.303807 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:50.304981 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:50.305378 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:50.305481 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:50.310225 (+ 4744us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.131:33931: BlockingRecv error: recv got EOF from 127.9.221.131:33931 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 11 12:03:50 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.129@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:50 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.129@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:03:50.458983 24968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:50.450489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.129:37305)
0811 12:03:50.451025 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:50.451041 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:50.451152 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:50.451431 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:50.451434 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:50.451892 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:50.452057 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:50.452066 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:50.453046 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:50.453049 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:50.453977 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:50.453987 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:50.454632 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:50.454636 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:50.454760 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:50.455402 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:50.455437 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:50.458698 (+ 3261us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":311,"threads_started":1}
W20260811 12:03:50.459169 24944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:50.450532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:37305 (local address 127.9.221.190:44917)
0811 12:03:50.450711 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:50.450715 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:50.451108 (+ 393us) server_negotiation.cc:407] Connection header received
0811 12:03:50.451244 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:50.451248 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:50.451306 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:50.451384 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:50.452179 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:50.452914 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:50.454099 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:50.454502 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:50.454594 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:50.458894 (+ 4300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.129:37305: BlockingRecv error: recv got EOF from 127.9.221.129:37305 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 11 12:03:50 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.130@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:50 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.130@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:03:50.912667 24970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:50.902796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.130:37049)
0811 12:03:50.903373 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:50.903391 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:50.903561 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:50.903981 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:50.903988 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:50.904473 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:50.904671 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:50.904681 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:50.905761 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:50.905774 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:50.906772 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:50.906785 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:50.907613 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:50.907621 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:50.907876 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:50.908465 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:50.908485 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:50.912411 (+ 3926us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":308,"threads_started":1}
W20260811 12:03:50.912786 24944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:50.902902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:37049 (local address 127.9.221.190:44917)
0811 12:03:50.903151 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:50.903160 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:50.903480 (+ 320us) server_negotiation.cc:407] Connection header received
0811 12:03:50.903733 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:50.903741 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:50.903827 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:50.903945 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:50.904859 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:50.905587 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:50.906933 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:50.907426 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:50.907575 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:50.912600 (+ 5025us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.130:37049: BlockingRecv error: recv got EOF from 127.9.221.130:37049 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 11 12:03:51 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.131@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:51 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.131@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:03:51.320436 24971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:51.311520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.131:40867)
0811 12:03:51.312034 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:51.312051 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:51.312165 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:51.312475 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:51.312478 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:51.312881 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:51.313054 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:51.313062 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:51.314031 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:51.314035 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:51.315073 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:51.315084 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:51.316020 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:51.316024 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:51.316197 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:51.316867 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:51.316884 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:51.320208 (+ 3324us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":319,"threads_started":1}
W20260811 12:03:51.320674 24944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:51.311681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:40867 (local address 127.9.221.190:44917)
0811 12:03:51.311938 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:51.311945 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:51.312132 (+ 187us) server_negotiation.cc:407] Connection header received
0811 12:03:51.312261 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:51.312265 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:51.312339 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:51.312422 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:51.313170 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:51.313912 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:51.315219 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:51.315834 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:51.315981 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:51.320394 (+ 4413us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.131:40867: BlockingRecv error: recv got EOF from 127.9.221.131:40867 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 11 12:03:51 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.129@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:51 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.129@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:03:51.469480 24972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:51.460156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.129:50913)
0811 12:03:51.460690 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:51.460705 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:51.460864 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:51.461205 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:51.461209 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:51.461647 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:51.461858 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:51.461866 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:51.462964 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:51.462976 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:51.464035 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:51.464065 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:51.464918 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:51.464923 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:51.465107 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:51.465713 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:51.465730 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:51.469117 (+ 3387us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":245,"threads_started":1}
W20260811 12:03:51.469543 24944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:51.460223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:50913 (local address 127.9.221.190:44917)
0811 12:03:51.460437 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:51.460445 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:51.460793 (+ 348us) server_negotiation.cc:407] Connection header received
0811 12:03:51.460995 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:51.461002 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:51.461069 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:51.461174 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:51.462009 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:51.462813 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:51.464212 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:51.464763 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:51.464903 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:51.469314 (+ 4411us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.129:50913: BlockingRecv error: recv got EOF from 127.9.221.129:50913 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 11 12:03:51 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.130@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:51 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.130@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:03:51.922642 24974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:51.913823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.130:59647)
0811 12:03:51.914484 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:51.914500 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:51.914667 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:51.914969 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:51.914973 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:51.915499 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:51.915755 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:51.915765 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:51.916762 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:51.916768 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:51.917723 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:51.917734 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:51.918424 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:51.918428 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:51.918538 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:51.919078 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:51.919098 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:51.922415 (+ 3317us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":308,"threads_started":1}
W20260811 12:03:51.922850 24944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:51.913855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:59647 (local address 127.9.221.190:44917)
0811 12:03:51.914042 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:51.914046 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:51.914575 (+ 529us) server_negotiation.cc:407] Connection header received
0811 12:03:51.914783 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:51.914786 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:51.914846 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:51.914930 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:51.915881 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:51.916627 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:51.917847 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:51.918296 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:51.918390 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:51.922610 (+ 4220us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.130:59647: BlockingRecv error: recv got EOF from 127.9.221.130:59647 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 11 12:03:52 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.131@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:52 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.131@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:03:52.330495 24976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:52.321340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.131:41387)
0811 12:03:52.322016 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:52.322033 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:52.322146 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:52.322462 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:52.322465 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:52.322865 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:52.323052 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:52.323061 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:52.324107 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:52.324110 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:52.325054 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:52.325065 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:52.325730 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:52.325734 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:52.325846 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:52.326386 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:52.326402 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:52.330244 (+ 3842us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":274,"threads_started":1}
W20260811 12:03:52.330643 24944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:52.321412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:41387 (local address 127.9.221.190:44917)
0811 12:03:52.321776 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:52.321786 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:52.322120 (+ 334us) server_negotiation.cc:407] Connection header received
0811 12:03:52.322245 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:52.322249 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:52.322327 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:52.322410 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:52.323174 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:52.323996 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:52.325175 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:52.325604 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:52.325697 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:52.330418 (+ 4721us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.131:41387: BlockingRecv error: recv got EOF from 127.9.221.131:41387 (error 108)
Metrics: {"server-negotiator.queue_time_us":188}
Aug 11 12:03:52 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.129@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:52 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.129@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:03:52.479598 24977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:52.470657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.129:46649)
0811 12:03:52.471262 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:52.471278 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:52.471419 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:52.471899 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:52.471905 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:52.472466 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:52.472727 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:52.472742 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:52.473728 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:52.473731 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:52.474649 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:52.474660 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:52.475331 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:52.475335 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:52.475446 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:52.476122 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:52.476140 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:52.479363 (+ 3223us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":311,"threads_started":1}
W20260811 12:03:52.479876 24944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:52.470809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:46649 (local address 127.9.221.190:44917)
0811 12:03:52.471056 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:52.471062 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:52.471378 (+ 316us) server_negotiation.cc:407] Connection header received
0811 12:03:52.471544 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:52.471549 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:52.471689 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:52.471815 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:52.472888 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:52.473617 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:52.474785 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:52.475184 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:52.475280 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:52.479569 (+ 4289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.129:46649: BlockingRecv error: recv got EOF from 127.9.221.129:46649 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 11 12:03:52 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.130@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:52 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.130@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:03:52.936928 24978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:52.923796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.130:56863)
0811 12:03:52.924378 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:52.924400 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:52.924546 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:52.924972 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:52.924976 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:52.925552 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:52.925728 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:52.925737 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:52.926990 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:52.926994 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:52.928032 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:52.928045 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:52.928856 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:52.928860 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:52.928977 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:52.932543 (+ 3566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:52.932562 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:52.936545 (+ 3983us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":424,"threads_started":1}
W20260811 12:03:52.937142 24944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:52.923893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:56863 (local address 127.9.221.190:44917)
0811 12:03:52.924096 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:52.924104 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:52.924491 (+ 387us) server_negotiation.cc:407] Connection header received
0811 12:03:52.924666 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:52.924673 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:52.924787 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:52.924923 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:52.925971 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:52.926855 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:52.928161 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:52.928694 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:52.928793 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:52.936882 (+ 8089us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.130:56863: BlockingRecv error: recv got EOF from 127.9.221.130:56863 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 11 12:03:53 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.131@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:53 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.131@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:03:53.340421 24979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:53.331543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.131:51887)
0811 12:03:53.332080 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:53.332096 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:53.332209 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:53.332494 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:53.332498 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:53.332971 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:53.333144 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:53.333153 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:53.334164 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:53.334168 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:53.335167 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:53.335179 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:53.335930 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:53.335934 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:53.336058 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:53.336759 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:53.336777 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:53.340120 (+ 3343us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":305,"threads_started":1}
W20260811 12:03:53.340631 24944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:53.331664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:51887 (local address 127.9.221.190:44917)
0811 12:03:53.331896 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:53.331902 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:53.332175 (+ 273us) server_negotiation.cc:407] Connection header received
0811 12:03:53.332308 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:53.332312 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:53.332373 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:53.332455 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:53.333264 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:53.334008 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:53.335292 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:53.335775 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:53.335868 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:53.340373 (+ 4505us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.131:51887: BlockingRecv error: recv got EOF from 127.9.221.131:51887 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 11 12:03:53 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.129@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:53 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.129@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:03:53.490110 24980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:53.480727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.129:59733)
0811 12:03:53.481282 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:53.481299 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:53.481411 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:53.481829 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:53.481832 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:53.482263 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:53.482407 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:53.482416 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:53.483773 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:53.483779 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:53.484727 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:53.484738 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:53.485467 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:53.485473 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:53.485644 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:53.486268 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:53.486286 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:53.489878 (+ 3592us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":302,"threads_started":1}
W20260811 12:03:53.490290 24944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:53.480782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:59733 (local address 127.9.221.190:44917)
0811 12:03:53.480974 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:53.480979 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:53.481375 (+ 396us) server_negotiation.cc:407] Connection header received
0811 12:03:53.481588 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:53.481593 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:53.481690 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:53.481788 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:53.482588 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:53.483620 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:53.484882 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:53.485295 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:53.485427 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:53.490070 (+ 4643us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.129:59733: BlockingRecv error: recv got EOF from 127.9.221.129:59733 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 11 12:03:53 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.130@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:53 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.130@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:03:53.949383 24981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:53.938154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.130:57337)
0811 12:03:53.938772 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:53.938789 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:53.938897 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:53.939210 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:53.939214 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:53.939692 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:53.939855 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:53.939863 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:53.940877 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:53.940880 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:53.941827 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:53.941838 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:53.942533 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:53.942537 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:53.942659 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:53.943356 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:53.943393 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:53.949063 (+ 5670us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":346,"threads_started":1}
W20260811 12:03:53.955971 24944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:53.938261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:57337 (local address 127.9.221.190:44917)
0811 12:03:53.938470 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:53.938474 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:53.938898 (+ 424us) server_negotiation.cc:407] Connection header received
0811 12:03:53.939009 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:53.939013 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:53.939079 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:53.939163 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:53.939987 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:53.940746 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:53.941959 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:53.942401 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:53.942505 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:53.950130 (+ 7625us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.130:57337: BlockingRecv error: recv got EOF from 127.9.221.130:57337 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 11 12:03:54 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.131@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:54 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.131@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:03:54.351857 24983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:54.341692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.131:47687)
0811 12:03:54.342236 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:54.342252 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:54.342410 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:54.342774 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:54.342779 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:54.343229 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:54.343376 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:54.343384 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:54.344528 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:54.344536 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:54.345492 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:54.345503 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:54.346331 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:54.346338 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:54.346581 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:54.347360 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:54.347389 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:54.351487 (+ 4098us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":271,"threads_started":1}
W20260811 12:03:54.351945 24944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:54.341712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:47687 (local address 127.9.221.190:44917)
0811 12:03:54.341882 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:54.341887 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:54.342342 (+ 455us) server_negotiation.cc:407] Connection header received
0811 12:03:54.342557 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:54.342562 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:54.342634 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:54.342738 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:54.343528 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:54.344373 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:54.345633 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:54.346085 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:54.346247 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:54.351713 (+ 5466us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.131:47687: BlockingRecv error: recv got EOF from 127.9.221.131:47687 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 11 12:03:54 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.129@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:54 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.129@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:03:54.501286 24984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:54.491043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.129:57051)
0811 12:03:54.491811 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:54.491835 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:54.491980 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:54.492361 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:54.492366 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:54.492801 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:54.492965 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:54.492974 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:54.494072 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:54.494078 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:54.495051 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:54.495063 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:54.495896 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:54.495901 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:54.496090 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:54.496800 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:54.496817 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:54.500910 (+ 4093us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":645,"thread_start_us":516,"threads_started":1}
W20260811 12:03:54.501394 24944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:54.491163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:57051 (local address 127.9.221.190:44917)
0811 12:03:54.491391 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:54.491400 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:54.491906 (+ 506us) server_negotiation.cc:407] Connection header received
0811 12:03:54.492119 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:54.492128 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:54.492209 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:54.492326 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:54.493125 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:54.493926 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:54.495203 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:54.495740 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:54.495885 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:54.501133 (+ 5248us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.129:57051: BlockingRecv error: recv got EOF from 127.9.221.129:57051 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 11 12:03:54 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.130@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:54 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.130@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:03:54.961439 24985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:54.951168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.130:45663)
0811 12:03:54.951680 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:54.951700 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:54.951818 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:54.952217 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:54.952221 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:54.952579 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:54.952733 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:54.952741 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:54.953929 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:54.953935 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:54.955609 (+ 1674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:54.955625 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:54.956283 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:54.956288 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:54.956416 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:54.957015 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:54.957033 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:54.961140 (+ 4107us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":252,"threads_started":1}
W20260811 12:03:54.961803 24944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:54.951557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:45663 (local address 127.9.221.190:44917)
0811 12:03:54.951774 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:54.951780 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:54.951803 (+ 23us) server_negotiation.cc:407] Connection header received
0811 12:03:54.951919 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:54.951925 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:54.952022 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:54.952142 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:54.952889 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:54.953639 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:54.955764 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:54.956142 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:54.956236 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:54.961444 (+ 5208us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.130:45663: BlockingRecv error: recv got EOF from 127.9.221.130:45663 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 11 12:03:55 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.131@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:55 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.131@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:03:55.362903 24986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:55.352973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.131:37517)
0811 12:03:55.353502 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:55.353518 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:55.353681 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:55.354235 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:55.354239 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:55.354795 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:55.354985 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:55.354994 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:55.356473 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:55.356477 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:55.357456 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:55.357467 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:55.358346 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:55.358350 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:55.358473 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:55.359128 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:55.359145 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:55.362628 (+ 3483us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":280,"threads_started":1}
W20260811 12:03:55.363718 24944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:55.353181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:37517 (local address 127.9.221.190:44917)
0811 12:03:55.353385 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:55.353393 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:55.353606 (+ 213us) server_negotiation.cc:407] Connection header received
0811 12:03:55.353824 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:55.353829 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:55.353901 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:55.354013 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:55.355157 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:55.356302 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:55.357596 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:55.358207 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:55.358341 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:55.362877 (+ 4536us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.131:37517: BlockingRecv error: recv got EOF from 127.9.221.131:37517 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 11 12:03:55 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.129@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:55 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.129@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 12:03:55.513437 24944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:55.502536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:49663 (local address 127.9.221.190:44917)
0811 12:03:55.502715 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:55.502723 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:55.503216 (+ 493us) server_negotiation.cc:407] Connection header received
0811 12:03:55.503373 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:55.503377 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:55.503444 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:55.503535 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:55.504568 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:55.505264 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:55.507042 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:55.507495 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:55.507588 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:55.513249 (+ 5661us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.129:49663: BlockingRecv error: recv got EOF from 127.9.221.129:49663 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 12:03:55.513540 24987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:55.502423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.129:49663)
0811 12:03:55.503102 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:55.503124 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:55.503267 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:55.503594 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:55.503600 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:55.504168 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:55.504399 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:55.504411 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:55.505411 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:55.505417 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:55.506903 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:55.506919 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:55.507679 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:55.507686 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:55.507879 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:55.508540 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:55.508564 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:55.513088 (+ 4524us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":384,"threads_started":1}
Aug 11 12:03:55 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.130@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:55 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.130@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:03:55.976114 24990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:55.962812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.130:39099)
0811 12:03:55.963428 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:55.963448 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:55.963573 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:55.964381 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:55.964390 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:55.965122 (+ 732us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:55.965454 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:55.965472 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:55.967267 (+ 1795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:55.967276 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:55.968987 (+ 1711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:55.969010 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:55.970041 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:55.970048 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:55.970324 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:55.971117 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:55.971187 (+ 70us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:55.975748 (+ 4561us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":337,"threads_started":1}
W20260811 12:03:55.976307 24944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:55.963219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:39099 (local address 127.9.221.190:44917)
0811 12:03:55.963493 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:55.963501 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:55.963523 (+ 22us) server_negotiation.cc:407] Connection header received
0811 12:03:55.963831 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:55.963841 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:55.963946 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:55.964063 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:55.965672 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:55.967021 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:55.969163 (+ 2142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:55.969873 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:55.970029 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:55.976001 (+ 5972us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.130:39099: BlockingRecv error: recv got EOF from 127.9.221.130:39099 (error 108)
Metrics: {"server-negotiator.queue_time_us":125}
Aug 11 12:03:56 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.131@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:56 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.131@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 12:03:56.374872 24944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:56.364229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:42625 (local address 127.9.221.190:44917)
0811 12:03:56.364436 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:56.364443 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:56.364763 (+ 320us) server_negotiation.cc:407] Connection header received
0811 12:03:56.364985 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:56.364990 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:56.365098 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:56.365203 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:56.366378 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:56.367780 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:56.368972 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:56.369761 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:56.369896 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:56.374676 (+ 4780us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.131:42625: BlockingRecv error: recv got EOF from 127.9.221.131:42625 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 12:03:56.374946 24991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:56.364143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.131:42625)
0811 12:03:56.364655 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:56.364677 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:56.364840 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:56.365274 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:56.365283 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:56.365905 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:56.366172 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:56.366187 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:56.367905 (+ 1718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:56.367909 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:56.368832 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:56.368843 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:56.369931 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:56.369936 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:56.370062 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:56.370703 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:56.370720 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:56.374513 (+ 3793us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":288,"threads_started":1}
Aug 11 12:03:56 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.129@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:56 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.129@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 12:03:56.524904 24944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:56.514745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:57115 (local address 127.9.221.190:44917)
0811 12:03:56.514963 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:56.514972 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:56.515404 (+ 432us) server_negotiation.cc:407] Connection header received
0811 12:03:56.515613 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:56.515627 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:56.515753 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:56.515883 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:56.516644 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:56.517392 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:56.518618 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:56.519090 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:56.519235 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:56.524693 (+ 5458us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.129:57115: BlockingRecv error: recv got EOF from 127.9.221.129:57115 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 12:03:56.524968 24993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:56.514717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.129:57115)
0811 12:03:56.515295 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:56.515312 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:56.515469 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:56.515893 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:56.515897 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:56.516327 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:56.516501 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:56.516510 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:56.517543 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:56.517550 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:56.518475 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:56.518486 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:56.519258 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:56.519263 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:56.519450 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:56.520203 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:56.520232 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:56.524537 (+ 4305us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":322,"threads_started":1}
Aug 11 12:03:56 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.130@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:56 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.130@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:03:56.986197 24994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:56.977218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.130:45789)
0811 12:03:56.977741 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:56.977758 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:56.977917 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:56.978264 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:56.978269 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:56.978689 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:56.978864 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:56.978873 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:56.979992 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:56.979998 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:56.980968 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:56.980979 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:56.981716 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:56.981722 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:56.981927 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:56.982554 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:56.982571 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:56.985966 (+ 3395us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":312,"threads_started":1}
W20260811 12:03:56.986375 24944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:56.977218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:45789 (local address 127.9.221.190:44917)
0811 12:03:56.977411 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:56.977419 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:56.977846 (+ 427us) server_negotiation.cc:407] Connection header received
0811 12:03:56.978043 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:56.978048 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:56.978122 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:56.978227 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:56.979023 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:56.979856 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:56.981113 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:56.981555 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:56.981704 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:56.986144 (+ 4440us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.130:45789: BlockingRecv error: recv got EOF from 127.9.221.130:45789 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 11 12:03:57 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.131@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:57 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.131@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:03:57.385667 24995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:57.375904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.131:49847)
0811 12:03:57.376537 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:57.376554 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:57.376674 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:57.376975 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:57.376979 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:57.377390 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:57.377583 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:57.377592 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:57.379035 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:57.379039 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:57.380124 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:57.380141 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:57.381169 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:57.381179 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:57.381328 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:57.382062 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:57.382080 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:57.385317 (+ 3237us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":362,"threads_started":1}
W20260811 12:03:57.385829 24944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:57.375965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:49847 (local address 127.9.221.190:44917)
0811 12:03:57.376141 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:57.376146 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:57.376634 (+ 488us) server_negotiation.cc:407] Connection header received
0811 12:03:57.376773 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:57.376776 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:57.376840 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:57.376923 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:57.377769 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:57.378904 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:57.380298 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:57.380964 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:57.381078 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:57.385558 (+ 4480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.131:49847: BlockingRecv error: recv got EOF from 127.9.221.131:49847 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 11 12:03:57 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.129@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:57 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.129@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:03:57.538553 24997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:57.525917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.129:52517)
0811 12:03:57.526478 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:57.526502 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:57.526660 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:57.527302 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:57.527307 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:57.527857 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:57.528086 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:57.528096 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:57.529527 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:57.529533 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:57.531141 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:57.531161 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:57.532458 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:57.532465 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:57.532684 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:57.533501 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:57.533531 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:57.538204 (+ 4673us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":362,"threads_started":1}
W20260811 12:03:57.538666 24944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:57.526245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:52517 (local address 127.9.221.190:44917)
0811 12:03:57.526480 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:57.526489 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:57.526574 (+ 85us) server_negotiation.cc:407] Connection header received
0811 12:03:57.526852 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:57.526860 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:57.526952 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:57.527058 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:57.528242 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:57.529346 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:57.531389 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:57.532278 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:57.532432 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:57.538404 (+ 5972us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.129:52517: BlockingRecv error: recv got EOF from 127.9.221.129:52517 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 11 12:03:57 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.130@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:57 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.130@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:03:58.001308 24998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:57.987344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.130:47473)
0811 12:03:57.987907 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:57.987922 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:57.988124 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:57.988554 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:57.988560 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:57.989022 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:57.989178 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:57.989186 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:57.990334 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:57.990356 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:57.991269 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:57.991280 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:57.991954 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:57.991958 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:57.992077 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:57.997041 (+ 4964us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:57.997059 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:58.001088 (+ 4029us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":342,"threads_started":1}
W20260811 12:03:58.001564 24944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:57.987499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:47473 (local address 127.9.221.190:44917)
0811 12:03:57.987738 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:57.987744 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:57.988010 (+ 266us) server_negotiation.cc:407] Connection header received
0811 12:03:57.988302 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:57.988307 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:57.988390 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:57.988511 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:57.989394 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:57.990217 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:57.991395 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:57.991826 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:57.991918 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:58.001286 (+ 9368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.130:47473: BlockingRecv error: recv got EOF from 127.9.221.130:47473 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 11 12:03:58 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.131@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:58 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.131@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:03:58.396358 24999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:58.386992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.131:37673)
0811 12:03:58.387466 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:58.387483 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:58.387595 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:58.387947 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:58.387950 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:58.388469 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:58.388613 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:58.388622 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:58.389819 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:58.389823 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:58.391225 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:58.391237 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:58.392135 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:58.392139 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:58.392262 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:58.392800 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:58.392817 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:58.396129 (+ 3312us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":256,"threads_started":1}
W20260811 12:03:58.396548 24944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:58.387018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:37673 (local address 127.9.221.190:44917)
0811 12:03:58.387234 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:58.387243 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:58.387566 (+ 323us) server_negotiation.cc:407] Connection header received
0811 12:03:58.387750 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:58.387753 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:58.387815 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:58.387897 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:58.388745 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:58.389671 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:58.391391 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:58.391993 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:58.392108 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:58.396332 (+ 4224us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.131:37673: BlockingRecv error: recv got EOF from 127.9.221.131:37673 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 11 12:03:58 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.129@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:58 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.129@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:03:58.549606 25001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:58.540003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.129:44323)
0811 12:03:58.540545 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:58.540567 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:58.540725 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:58.541081 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:58.541088 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:58.541538 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:58.541723 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:58.541732 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:58.542755 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:58.542761 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:58.543798 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:58.543810 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:58.544570 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:58.544576 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:58.544762 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:58.545522 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:58.545547 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:58.549361 (+ 3814us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":254,"threads_started":1}
W20260811 12:03:58.549816 24944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:58.540079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:44323 (local address 127.9.221.190:44917)
0811 12:03:58.540521 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:58.540529 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:58.540654 (+ 125us) server_negotiation.cc:407] Connection header received
0811 12:03:58.540857 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:58.540866 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:58.540927 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:58.541039 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:58.541866 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:58.542603 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:58.543943 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:58.544380 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:58.544548 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:58.549544 (+ 4996us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.129:44323: BlockingRecv error: recv got EOF from 127.9.221.129:44323 (error 108)
Metrics: {"server-negotiator.queue_time_us":269}
I20260811 12:03:58.985414 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 24314
I20260811 12:03:58.998039 25003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:58.986688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39648 (local address 127.9.221.129:33255)
0811 12:03:58.987403 (+ 715us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:58.987411 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:58.987461 (+ 50us) server_negotiation.cc:407] Connection header received
0811 12:03:58.987522 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:58.987528 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:58.987900 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:58.988079 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:58.989617 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:58.990839 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:58.992020 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:58.992588 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:58.992690 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:58.995150 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:58.995158 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:58.995170 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:58.995204 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:58.997524 (+ 2320us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.9.221.129@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 12:03:58.997802 (+ 278us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39648: Request ticket server kudu/127.9.221.129@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":406,"threads_started":1}
W20260811 12:03:58.998195 25003 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39648: Request ticket server kudu/127.9.221.129@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 12:03:58.998202 10102 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 24314: Not authorized: Client connection negotiation failed: client connection to 127.9.221.129:33255: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.9.221.129@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 12:03:58.998411 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 24314
Aug 11 12:03:59 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.130@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:03:59 dist-test-slave-mjjr krb5kdc[23464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.221.130@KRBTEST.COM for oryx/127.9.221.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:03:59.018680 25004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:59.003266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44917 (local address 127.9.221.130:50723)
0811 12:03:59.009252 (+ 5986us) negotiation.cc:107] Waiting for socket to connect
0811 12:03:59.009272 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:03:59.009414 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:03:59.009762 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:03:59.009766 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:03:59.010137 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:03:59.010311 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:59.010320 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:59.011937 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:59.011941 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:59.012839 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:03:59.012850 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:03:59.013912 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:03:59.013916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:03:59.014047 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:59.014686 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:03:59.014702 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:03:59.018414 (+ 3712us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.221.190:44917: Server oryx/127.9.221.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":5822,"thread_start_us":5687,"threads_started":1}
W20260811 12:03:59.019049 24944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:03:59.004144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:50723 (local address 127.9.221.190:44917)
0811 12:03:59.007930 (+ 3786us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:59.007939 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:59.009395 (+ 1456us) server_negotiation.cc:407] Connection header received
0811 12:03:59.009517 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:59.009525 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:59.009617 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:59.009726 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:59.010469 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:59.011777 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:59.012985 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:59.013765 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:59.013924 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:59.018721 (+ 4797us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.221.130:50723: BlockingRecv error: recv got EOF from 127.9.221.130:50723 (error 108)
Metrics: {"server-negotiator.queue_time_us":3620}
I20260811 12:03:59.029152 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 24530
I20260811 12:03:59.041319 25005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:59.030218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50978 (local address 127.9.221.130:41735)
0811 12:03:59.030831 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:59.030838 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:59.030865 (+ 27us) server_negotiation.cc:407] Connection header received
0811 12:03:59.030921 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:59.030934 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:59.031156 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:59.031371 (+ 215us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:59.032761 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:59.033865 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:59.035076 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:59.035831 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:59.035972 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:59.038339 (+ 2367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:59.038347 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:59.038359 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:59.038399 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:59.040823 (+ 2424us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.9.221.130@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 12:03:59.041093 (+ 270us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50978: Request ticket server kudu/127.9.221.130@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":335,"threads_started":1}
W20260811 12:03:59.041407 10102 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 24530: Not authorized: Client connection negotiation failed: client connection to 127.9.221.130:41735: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.9.221.130@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 12:03:59.041451 25005 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50978: Request ticket server kudu/127.9.221.130@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 12:03:59.041603 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 24530
I20260811 12:03:59.066327 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 24748
I20260811 12:03:59.079046 25006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:59.067610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50518 (local address 127.9.221.131:43925)
0811 12:03:59.068209 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:59.068217 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:59.068244 (+ 27us) server_negotiation.cc:407] Connection header received
0811 12:03:59.068309 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:59.068315 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:59.068556 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:59.068772 (+ 216us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:59.070221 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:59.071259 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:59.072576 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:59.073147 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:59.073298 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:59.075956 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:59.075964 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:59.075978 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:03:59.076016 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:59.078328 (+ 2312us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.9.221.131@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 12:03:59.078697 (+ 369us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50518: Request ticket server kudu/127.9.221.131@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":290,"threads_started":1}
W20260811 12:03:59.079094 10102 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 24748: Not authorized: Client connection negotiation failed: client connection to 127.9.221.131:43925: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.9.221.131@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 12:03:59.079173 25006 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50518: Request ticket server kudu/127.9.221.131@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 12:03:59.079262 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 24748
I20260811 12:03:59.104130 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 24220
I20260811 12:03:59.105557 24304 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:03:59.225378 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 24220
2026-08-11T12:03:59Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21591 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-11T12:03:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:03:59Z Disabled control of system clock
I20260811 12:03:59.318282 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:38331
--webserver_interface=127.9.221.190
--webserver_port=0
--builtin_ntp_servers=127.9.221.148:46783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:38331
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 12:03:59.651962 25013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:03:59.652235 25013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:03:59.652364 25013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:03:59.659355 25013 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:03:59.659446 25013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:03:59.659487 25013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:03:59.659582 25013 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:03:59.659674 25013 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:03:59.668287 25013 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:46783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:38331
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:38331
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.221.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:03:59.669540 25013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:03:59.671223 25013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:03:59.683434 25020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:59.683406 25022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:03:59.683729 25024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:03:59.684262 25013 server_base.cc:1034] running on GCE node
I20260811 12:03:59.685041 25013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:03:59.686353 25013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:03:59.687592 25013 hybrid_clock.cc:648] HybridClock initialized: now 1786449839687553 us; error 63 us; skew 500 ppm
I20260811 12:03:59.687965 25013 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:03:59.690114 25013 webserver.cc:460] Webserver started at http://127.9.221.190:33555/ using document root <none> and password file <none>
I20260811 12:03:59.690820 25013 fs_manager.cc:359] Metadata directory not provided
I20260811 12:03:59.690968 25013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:03:59.691354 25013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:03:59.694157 25013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "5d8c3d8e1f364847b041dd1102dcf70c"
format_stamp: "Formatted at 2026-08-11 12:03:59 on dist-test-slave-mjjr"
server_key: "7dc8e7b6c448ce128dcf928b79a7dff8"
server_key_iv: "ea575e64d15e2ff573371aa6b94e505b"
server_key_version: "encryptionkey@0"
I20260811 12:03:59.694936 25013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "5d8c3d8e1f364847b041dd1102dcf70c"
format_stamp: "Formatted at 2026-08-11 12:03:59 on dist-test-slave-mjjr"
server_key: "7dc8e7b6c448ce128dcf928b79a7dff8"
server_key_iv: "ea575e64d15e2ff573371aa6b94e505b"
server_key_version: "encryptionkey@0"
I20260811 12:03:59.700402 25013 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 12:03:59.704420 25031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:03:59.705672 25013 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:03:59.705857 25013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "5d8c3d8e1f364847b041dd1102dcf70c"
format_stamp: "Formatted at 2026-08-11 12:03:59 on dist-test-slave-mjjr"
server_key: "7dc8e7b6c448ce128dcf928b79a7dff8"
server_key_iv: "ea575e64d15e2ff573371aa6b94e505b"
server_key_version: "encryptionkey@0"
I20260811 12:03:59.706079 25013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:03:59.724798 25013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:03:59.725575 25013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:03:59.725869 25013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:03:59.744320 25013 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.190:38331
I20260811 12:03:59.744354 25098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.190:38331 every 8 connection(s)
I20260811 12:03:59.745559 25013 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:03:59.749197 25100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:03:59.750448 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 25013
I20260811 12:03:59.750666 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:03:59.751365 10102 external_mini_cluster.cc:1428] Setting key 57e2cd9cee62e438a7e5b8a1538df5d2
I20260811 12:03:59.765496 25100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d8c3d8e1f364847b041dd1102dcf70c: Bootstrap starting.
I20260811 12:03:59.769553 25100 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d8c3d8e1f364847b041dd1102dcf70c: Neither blocks nor log segments found. Creating new log.
I20260811 12:03:59.769660 25104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:03:59.753603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55986 (local address 127.9.221.190:38331)
0811 12:03:59.754844 (+ 1241us) server_negotiation.cc:206] Beginning negotiation
0811 12:03:59.754857 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 12:03:59.754912 (+ 55us) server_negotiation.cc:407] Connection header received
0811 12:03:59.756698 (+ 1786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:03:59.756725 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:03:59.757354 (+ 629us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:03:59.757797 (+ 443us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:03:59.758526 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:59.760697 (+ 2171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:59.763072 (+ 2375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:03:59.764033 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:03:59.764225 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:03:59.766450 (+ 2225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:03:59.766481 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:03:59.766497 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:03:59.767213 (+ 716us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:03:59.767328 (+ 115us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:03:59.767357 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:03:59.767482 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 12:03:59.768115 (+ 633us) server_negotiation.cc:299] Negotiation successful
0811 12:03:59.768416 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":686,"thread_start_us":416,"threads_started":1}
I20260811 12:03:59.770742 25100 log.cc:824] T 00000000000000000000000000000000 P 5d8c3d8e1f364847b041dd1102dcf70c: Log is configured to *not* fsync() on all Append() calls
I20260811 12:03:59.773553 25100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d8c3d8e1f364847b041dd1102dcf70c: No bootstrap required, opened a new log
I20260811 12:03:59.780539 25100 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d8c3d8e1f364847b041dd1102dcf70c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d8c3d8e1f364847b041dd1102dcf70c" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38331 } }
I20260811 12:03:59.780892 25100 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d8c3d8e1f364847b041dd1102dcf70c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:03:59.781009 25100 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d8c3d8e1f364847b041dd1102dcf70c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d8c3d8e1f364847b041dd1102dcf70c, State: Initialized, Role: FOLLOWER
I20260811 12:03:59.781512 25100 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d8c3d8e1f364847b041dd1102dcf70c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d8c3d8e1f364847b041dd1102dcf70c" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38331 } }
I20260811 12:03:59.781752 25100 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d8c3d8e1f364847b041dd1102dcf70c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:03:59.781898 25100 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d8c3d8e1f364847b041dd1102dcf70c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:03:59.782238 25100 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d8c3d8e1f364847b041dd1102dcf70c [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:03:59.784711 25100 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d8c3d8e1f364847b041dd1102dcf70c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d8c3d8e1f364847b041dd1102dcf70c" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38331 } }
I20260811 12:03:59.785254 25100 leader_election.cc:302] T 00000000000000000000000000000000 P 5d8c3d8e1f364847b041dd1102dcf70c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d8c3d8e1f364847b041dd1102dcf70c; no voters:
I20260811 12:03:59.785844 25100 leader_election.cc:288] T 00000000000000000000000000000000 P 5d8c3d8e1f364847b041dd1102dcf70c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:03:59.785949 25106 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d8c3d8e1f364847b041dd1102dcf70c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:03:59.786790 25106 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d8c3d8e1f364847b041dd1102dcf70c [term 1 LEADER]: Becoming Leader. State: Replica: 5d8c3d8e1f364847b041dd1102dcf70c, State: Running, Role: LEADER
I20260811 12:03:59.787418 25106 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d8c3d8e1f364847b041dd1102dcf70c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d8c3d8e1f364847b041dd1102dcf70c" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38331 } }
I20260811 12:03:59.788153 25100 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d8c3d8e1f364847b041dd1102dcf70c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:03:59.790377 25110 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d8c3d8e1f364847b041dd1102dcf70c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d8c3d8e1f364847b041dd1102dcf70c. Latest consensus state: current_term: 1 leader_uuid: "5d8c3d8e1f364847b041dd1102dcf70c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d8c3d8e1f364847b041dd1102dcf70c" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38331 } } }
I20260811 12:03:59.790822 25110 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d8c3d8e1f364847b041dd1102dcf70c [sys.catalog]: This master's current role is: LEADER
I20260811 12:03:59.791689 25107 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d8c3d8e1f364847b041dd1102dcf70c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d8c3d8e1f364847b041dd1102dcf70c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d8c3d8e1f364847b041dd1102dcf70c" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 38331 } } }
I20260811 12:03:59.792161 25107 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d8c3d8e1f364847b041dd1102dcf70c [sys.catalog]: This master's current role is: LEADER
I20260811 12:03:59.792775 25113 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:03:59.799338 25113 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:03:59.808596 25113 catalog_manager.cc:1347] Generated new cluster ID: 87020e6eb99e45a29d75e9c3fd5f62a7
I20260811 12:03:59.808719 25113 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:03:59.825070 25113 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:03:59.826220 25113 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:03:59.834379 25113 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d8c3d8e1f364847b041dd1102dcf70c: Generated new TSK 0
I20260811 12:03:59.835114 25113 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:03:59.840526 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.129:0
--local_ip_for_outbound_sockets=127.9.221.129
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:38331
--builtin_ntp_servers=127.9.221.148:46783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 12:04:00.180049 25131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:04:00.180346 25131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:04:00.180503 25131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:04:00.188042 25131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:04:00.188149 25131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:04:00.188309 25131 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.129
I20260811 12:04:00.197010 25131 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:46783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:38331
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.129
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:04:00.198185 25131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:04:00.199977 25131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:04:00.213043 25137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:04:00.213109 25144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:04:00.213160 25139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:04:00.214255 25131 server_base.cc:1034] running on GCE node
I20260811 12:04:00.214846 25131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:04:00.215934 25131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:04:00.217151 25131 hybrid_clock.cc:648] HybridClock initialized: now 1786449840217094 us; error 56 us; skew 500 ppm
I20260811 12:04:00.217465 25131 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:04:00.219732 25131 webserver.cc:460] Webserver started at http://127.9.221.129:34519/ using document root <none> and password file <none>
I20260811 12:04:00.220389 25131 fs_manager.cc:359] Metadata directory not provided
I20260811 12:04:00.220549 25131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:04:00.220865 25131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:04:00.223552 25131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/data/instance:
uuid: "c274b66f8a364ae1bfd3ce2b0f080375"
format_stamp: "Formatted at 2026-08-11 12:04:00 on dist-test-slave-mjjr"
server_key: "3b0660f5a2575f552c597c854af94f8b"
server_key_iv: "4b0a3e0aa2e385a18f8d7ebca0137301"
server_key_version: "encryptionkey@0"
I20260811 12:04:00.224378 25131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance:
uuid: "c274b66f8a364ae1bfd3ce2b0f080375"
format_stamp: "Formatted at 2026-08-11 12:04:00 on dist-test-slave-mjjr"
server_key: "3b0660f5a2575f552c597c854af94f8b"
server_key_iv: "4b0a3e0aa2e385a18f8d7ebca0137301"
server_key_version: "encryptionkey@0"
I20260811 12:04:00.229339 25131 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:04:00.232586 25151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:04:00.233786 25131 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 12:04:00.234011 25131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/wal
uuid: "c274b66f8a364ae1bfd3ce2b0f080375"
format_stamp: "Formatted at 2026-08-11 12:04:00 on dist-test-slave-mjjr"
server_key: "3b0660f5a2575f552c597c854af94f8b"
server_key_iv: "4b0a3e0aa2e385a18f8d7ebca0137301"
server_key_version: "encryptionkey@0"
I20260811 12:04:00.234222 25131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:04:00.254580 25131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:04:00.255352 25131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:04:00.255692 25131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:04:00.256588 25131 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:04:00.257994 25131 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:04:00.258119 25131 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:04:00.258236 25131 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:04:00.258322 25131 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:04:00.292452 25131 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.129:40663
I20260811 12:04:00.292485 25312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.129:40663 every 8 connection(s)
I20260811 12:04:00.293710 25131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
I20260811 12:04:00.298717 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 25131
I20260811 12:04:00.298933 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance
I20260811 12:04:00.299564 10102 external_mini_cluster.cc:1428] Setting key 112c4adf887d757f067356af60d365a1
I20260811 12:04:00.302659 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.130:0
--local_ip_for_outbound_sockets=127.9.221.130
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:38331
--builtin_ntp_servers=127.9.221.148:46783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 12:04:00.307049 25320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:00.296533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:49555 (local address 127.9.221.190:38331)
0811 12:04:00.297001 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0811 12:04:00.297007 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:04:00.298320 (+ 1313us) server_negotiation.cc:407] Connection header received
0811 12:04:00.300082 (+ 1762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:04:00.300087 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:04:00.300171 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:04:00.300311 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:04:00.301309 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:00.302343 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:00.304015 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:00.304700 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:00.304839 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:00.306123 (+ 1284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:04:00.306131 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:04:00.306138 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:04:00.306163 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:04:00.306218 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:04:00.306237 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:04:00.306330 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 12:04:00.306531 (+ 201us) server_negotiation.cc:299] Negotiation successful
0811 12:04:00.306635 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":211,"threads_started":1}
I20260811 12:04:00.307829 25322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:00.297166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:38331 (local address 127.9.221.129:49555)
0811 12:04:00.298056 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0811 12:04:00.298091 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 12:04:00.299802 (+ 1711us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:04:00.300532 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:04:00.300539 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:04:00.300614 (+ 75us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:04:00.301147 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:04:00.301163 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:04:00.302512 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:04:00.302516 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:04:00.303877 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:04:00.303889 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:04:00.304905 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:04:00.304909 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:04:00.305112 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:00.305794 (+ 682us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:04:00.305796 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:04:00.305832 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:04:00.305908 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:04:00.306334 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:04:00.306342 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:04:00.306348 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 12:04:00.306495 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 12:04:00.306759 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":280,"threads_started":1}
I20260811 12:04:00.309331 25313 heartbeater.cc:347] Connected to a master server at 127.9.221.190:38331
I20260811 12:04:00.309633 25313 heartbeater.cc:464] Registering TS with master...
I20260811 12:04:00.310364 25313 heartbeater.cc:511] Master 127.9.221.190:38331 requested a full tablet report, sending...
I20260811 12:04:00.312359 25052 ts_manager.cc:194] Registered new tserver with Master: c274b66f8a364ae1bfd3ce2b0f080375 (127.9.221.129:40663)
I20260811 12:04:00.313846 25052 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.221.129:49555
W20260811 12:04:00.649999 25323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:04:00.650295 25323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:04:00.650457 25323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:04:00.657625 25323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:04:00.657740 25323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:04:00.657893 25323 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.130
I20260811 12:04:00.666750 25323 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:46783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:38331
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.130
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:04:00.668076 25323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:04:00.669878 25323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:04:00.684597 25331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:04:00.685040 25339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:04:00.685240 25329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:04:00.686271 25323 server_base.cc:1034] running on GCE node
I20260811 12:04:00.686880 25323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:04:00.687858 25323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:04:00.689123 25323 hybrid_clock.cc:648] HybridClock initialized: now 1786449840689024 us; error 57 us; skew 500 ppm
I20260811 12:04:00.689532 25323 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:04:00.692181 25323 webserver.cc:460] Webserver started at http://127.9.221.130:41435/ using document root <none> and password file <none>
I20260811 12:04:00.692853 25323 fs_manager.cc:359] Metadata directory not provided
I20260811 12:04:00.692996 25323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:04:00.693321 25323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:04:00.696376 25323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/data/instance:
uuid: "144a5154ba8f466983dc58e8f6d684ef"
format_stamp: "Formatted at 2026-08-11 12:04:00 on dist-test-slave-mjjr"
server_key: "ff7a3fc7ac8ff2581318163bc224f169"
server_key_iv: "0856fb2ef84e2d03e08b19e16cecbbe2"
server_key_version: "encryptionkey@0"
I20260811 12:04:00.697156 25323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance:
uuid: "144a5154ba8f466983dc58e8f6d684ef"
format_stamp: "Formatted at 2026-08-11 12:04:00 on dist-test-slave-mjjr"
server_key: "ff7a3fc7ac8ff2581318163bc224f169"
server_key_iv: "0856fb2ef84e2d03e08b19e16cecbbe2"
server_key_version: "encryptionkey@0"
I20260811 12:04:00.702848 25323 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:04:00.707460 25345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:04:00.708923 25323 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:04:00.709100 25323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/wal
uuid: "144a5154ba8f466983dc58e8f6d684ef"
format_stamp: "Formatted at 2026-08-11 12:04:00 on dist-test-slave-mjjr"
server_key: "ff7a3fc7ac8ff2581318163bc224f169"
server_key_iv: "0856fb2ef84e2d03e08b19e16cecbbe2"
server_key_version: "encryptionkey@0"
I20260811 12:04:00.709304 25323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:04:00.722497 25323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:04:00.723261 25323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:04:00.723574 25323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:04:00.724471 25323 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:04:00.725783 25323 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:04:00.725908 25323 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:04:00.726051 25323 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:04:00.726143 25323 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:04:00.763784 25323 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.130:44195
I20260811 12:04:00.763836 25528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.130:44195 every 8 connection(s)
I20260811 12:04:00.765027 25323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
I20260811 12:04:00.771286 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 25323
I20260811 12:04:00.771479 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance
I20260811 12:04:00.771998 10102 external_mini_cluster.cc:1428] Setting key d55015ed86a5d87239323c11e80edb43
I20260811 12:04:00.774755 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.131:0
--local_ip_for_outbound_sockets=127.9.221.131
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:38331
--builtin_ntp_servers=127.9.221.148:46783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 12:04:00.777398 25320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:00.767549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:59607 (local address 127.9.221.190:38331)
0811 12:04:00.767781 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 12:04:00.767786 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:04:00.769101 (+ 1315us) server_negotiation.cc:407] Connection header received
0811 12:04:00.770701 (+ 1600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:04:00.770705 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:04:00.770768 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:04:00.770852 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:04:00.771968 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:00.772914 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:00.774590 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:00.775093 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:00.775221 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:00.776619 (+ 1398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:04:00.776624 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:04:00.776627 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:04:00.776642 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:04:00.776672 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:04:00.776683 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:04:00.776774 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 12:04:00.777011 (+ 237us) server_negotiation.cc:299] Negotiation successful
0811 12:04:00.777103 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 12:04:00.778281 25533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:00.768094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:38331 (local address 127.9.221.130:59607)
0811 12:04:00.768963 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0811 12:04:00.768998 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 12:04:00.770486 (+ 1488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:04:00.771035 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:04:00.771042 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:04:00.771116 (+ 74us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:04:00.771809 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:04:00.771827 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:04:00.773101 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:04:00.773105 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:04:00.774427 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:04:00.774440 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:04:00.775302 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:04:00.775309 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:04:00.775557 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:00.776305 (+ 748us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:04:00.776308 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:04:00.776343 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:04:00.776421 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:04:00.776776 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:04:00.776789 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:04:00.776799 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 12:04:00.776981 (+ 182us) client_negotiation.cc:240] Negotiation successful
0811 12:04:00.777318 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":278,"threads_started":1}
I20260811 12:04:00.779690 25529 heartbeater.cc:347] Connected to a master server at 127.9.221.190:38331
I20260811 12:04:00.779999 25529 heartbeater.cc:464] Registering TS with master...
I20260811 12:04:00.780757 25529 heartbeater.cc:511] Master 127.9.221.190:38331 requested a full tablet report, sending...
I20260811 12:04:00.782197 25052 ts_manager.cc:194] Registered new tserver with Master: 144a5154ba8f466983dc58e8f6d684ef (127.9.221.130:44195)
I20260811 12:04:00.783393 25052 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.221.130:59607
W20260811 12:04:01.104537 25537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:04:01.104796 25537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:04:01.104983 25537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:04:01.112231 25537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:04:01.112317 25537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:04:01.112529 25537 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.131
I20260811 12:04:01.121502 25537 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:46783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:38331
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.131
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:04:01.122794 25537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:04:01.124617 25537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:04:01.138019 25551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:04:01.138108 25545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:04:01.138108 25548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:04:01.138882 25537 server_base.cc:1034] running on GCE node
I20260811 12:04:01.139479 25537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:04:01.140476 25537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:04:01.141685 25537 hybrid_clock.cc:648] HybridClock initialized: now 1786449841141640 us; error 35 us; skew 500 ppm
I20260811 12:04:01.142022 25537 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:04:01.144363 25537 webserver.cc:460] Webserver started at http://127.9.221.131:32811/ using document root <none> and password file <none>
I20260811 12:04:01.145088 25537 fs_manager.cc:359] Metadata directory not provided
I20260811 12:04:01.145226 25537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:04:01.145586 25537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:04:01.148545 25537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/data/instance:
uuid: "ec9bbfe2d7ef4178bb0f29e7ce2a6400"
format_stamp: "Formatted at 2026-08-11 12:04:01 on dist-test-slave-mjjr"
server_key: "c998050917a7fcb3decb0a59900e0462"
server_key_iv: "4c1e16dbd023498c4c0a0560819b0403"
server_key_version: "encryptionkey@0"
I20260811 12:04:01.149340 25537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance:
uuid: "ec9bbfe2d7ef4178bb0f29e7ce2a6400"
format_stamp: "Formatted at 2026-08-11 12:04:01 on dist-test-slave-mjjr"
server_key: "c998050917a7fcb3decb0a59900e0462"
server_key_iv: "4c1e16dbd023498c4c0a0560819b0403"
server_key_version: "encryptionkey@0"
I20260811 12:04:01.154592 25537 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 12:04:01.158048 25557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:04:01.159317 25537 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 12:04:01.159487 25537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/wal
uuid: "ec9bbfe2d7ef4178bb0f29e7ce2a6400"
format_stamp: "Formatted at 2026-08-11 12:04:01 on dist-test-slave-mjjr"
server_key: "c998050917a7fcb3decb0a59900e0462"
server_key_iv: "4c1e16dbd023498c4c0a0560819b0403"
server_key_version: "encryptionkey@0"
I20260811 12:04:01.159729 25537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:04:01.176811 25537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:04:01.177662 25537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:04:01.178059 25537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:04:01.179013 25537 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:04:01.180548 25537 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:04:01.180663 25537 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:04:01.180805 25537 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:04:01.180912 25537 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:04:01.214735 25537 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.131:41783
I20260811 12:04:01.214814 25736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.131:41783 every 8 connection(s)
I20260811 12:04:01.216116 25537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
I20260811 12:04:01.222942 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 25537
I20260811 12:04:01.223162 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance
I20260811 12:04:01.223729 10102 external_mini_cluster.cc:1428] Setting key e3b22f233d8dd699f4e12073ba242e48
I20260811 12:04:01.228971 25320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:01.218631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:48263 (local address 127.9.221.190:38331)
0811 12:04:01.218805 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 12:04:01.218809 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:04:01.220182 (+ 1373us) server_negotiation.cc:407] Connection header received
0811 12:04:01.221923 (+ 1741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:04:01.221932 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:04:01.221995 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:04:01.222072 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:04:01.223077 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:01.224252 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:01.226341 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:01.226969 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:01.227072 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:01.228334 (+ 1262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:04:01.228339 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:04:01.228342 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:04:01.228357 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:04:01.228386 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:04:01.228397 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:04:01.228472 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 12:04:01.228661 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 12:04:01.228732 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 12:04:01.229854 25741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:01.219128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:38331 (local address 127.9.221.131:48263)
0811 12:04:01.220040 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0811 12:04:01.220083 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 12:04:01.221689 (+ 1606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:04:01.222257 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:04:01.222264 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:04:01.222337 (+ 73us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:04:01.222904 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:04:01.222918 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:04:01.224407 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:04:01.224412 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:04:01.226208 (+ 1796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:04:01.226223 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:04:01.227170 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:04:01.227175 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:04:01.227365 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:01.228077 (+ 712us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:04:01.228079 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:04:01.228115 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:04:01.228182 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:04:01.228490 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:04:01.228497 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:04:01.228503 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 12:04:01.228632 (+ 129us) client_negotiation.cc:240] Negotiation successful
0811 12:04:01.228885 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":288,"threads_started":1}
I20260811 12:04:01.231058 25737 heartbeater.cc:347] Connected to a master server at 127.9.221.190:38331
I20260811 12:04:01.231288 25737 heartbeater.cc:464] Registering TS with master...
I20260811 12:04:01.232024 25737 heartbeater.cc:511] Master 127.9.221.190:38331 requested a full tablet report, sending...
I20260811 12:04:01.233280 25052 ts_manager.cc:194] Registered new tserver with Master: ec9bbfe2d7ef4178bb0f29e7ce2a6400 (127.9.221.131:41783)
I20260811 12:04:01.234019 25052 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.221.131:48263
I20260811 12:04:01.238897 10102 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 12:04:01.245190 25320 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:04:01.243292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56000 (local address 127.9.221.190:38331)
0811 12:04:01.243457 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 12:04:01.243461 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:04:01.244104 (+ 643us) server_negotiation.cc:407] Connection header received
0811 12:04:01.244301 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:04:01.244306 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:04:01.244388 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:04:01.244489 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:04:01.244955 (+ 466us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56000: BlockingRecv error: recv got EOF from 127.0.0.1:56000 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 12:04:01.245718 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 25131
I20260811 12:04:01.253368 25750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:01.246908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60786 (local address 127.9.221.129:40663)
0811 12:04:01.247448 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0811 12:04:01.247455 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:04:01.247478 (+ 23us) server_negotiation.cc:407] Connection header received
0811 12:04:01.247532 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:04:01.247537 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:04:01.247832 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:04:01.247983 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:04:01.248357 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:01.249302 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:01.250503 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:01.251026 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:01.251130 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:01.252088 (+ 958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:04:01.252095 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:04:01.252105 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:04:01.252686 (+ 581us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:04:01.252730 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:04:01.252748 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:04:01.252832 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 12:04:01.252991 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 12:04:01.253090 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":282,"threads_started":1}
I20260811 12:04:01.254446 25306 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:04:01.335314 25313 heartbeater.cc:503] Master 127.9.221.190:38331 was elected leader, sending a full tablet report...
I20260811 12:04:01.336678 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 25131
W20260811 12:04:01.363057 25033 connection.cc:535] server connection from 127.9.221.129:49555 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 12:04:01.363479 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 25323
I20260811 12:04:01.371522 25752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:01.364436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52968 (local address 127.9.221.130:44195)
0811 12:04:01.365011 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0811 12:04:01.365019 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:04:01.365046 (+ 27us) server_negotiation.cc:407] Connection header received
0811 12:04:01.365114 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:04:01.365120 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:04:01.365329 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:04:01.365505 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:04:01.365939 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:01.366955 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:01.368298 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:01.368843 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:01.369003 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:01.370091 (+ 1088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:04:01.370100 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:04:01.370112 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:04:01.370694 (+ 582us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:04:01.370739 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:04:01.370765 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:04:01.370889 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 12:04:01.371041 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 12:04:01.371191 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":305,"threads_started":1}
I20260811 12:04:01.372798 25524 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:04:01.454419 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 25323
I20260811 12:04:01.476686 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 25537
I20260811 12:04:01.485064 25756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:01.477679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40178 (local address 127.9.221.131:41783)
0811 12:04:01.478291 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0811 12:04:01.478297 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:04:01.478342 (+ 45us) server_negotiation.cc:407] Connection header received
0811 12:04:01.478418 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:04:01.478430 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:04:01.478661 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:04:01.478832 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:04:01.479308 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:01.480495 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:01.481782 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:01.482322 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:01.482471 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:01.483590 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:04:01.483599 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:04:01.483611 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:04:01.484239 (+ 628us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:04:01.484289 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:04:01.484307 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:04:01.484438 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 12:04:01.484578 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 12:04:01.484719 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":331,"threads_started":1}
I20260811 12:04:01.486258 25731 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:04:01.568308 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 25537
I20260811 12:04:01.591711 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 25013
I20260811 12:04:01.592890 25092 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:04:01.697140 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 25013
2026-08-11T12:04:01Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2467 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-11T12:04:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:04:01Z Disabled control of system clock
I20260811 12:04:01.786527 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:43095
--webserver_interface=127.9.221.190
--webserver_port=0
--builtin_ntp_servers=127.9.221.148:41987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:43095
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 12:04:02.119166 25766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:04:02.119526 25766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:04:02.119741 25766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:04:02.127027 25766 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:04:02.127168 25766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:04:02.127269 25766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:04:02.127385 25766 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:04:02.127475 25766 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:04:02.136524 25766 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:41987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:43095
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:43095
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.221.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:04:02.137933 25766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:04:02.139812 25766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:04:02.151942 25779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:04:02.152051 25774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:04:02.152034 25776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:04:02.152877 25766 server_base.cc:1034] running on GCE node
I20260811 12:04:02.153674 25766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:04:02.154963 25766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:04:02.156215 25766 hybrid_clock.cc:648] HybridClock initialized: now 1786449842156177 us; error 48 us; skew 500 ppm
I20260811 12:04:02.156574 25766 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:04:02.158799 25766 webserver.cc:460] Webserver started at http://127.9.221.190:40841/ using document root <none> and password file <none>
I20260811 12:04:02.159492 25766 fs_manager.cc:359] Metadata directory not provided
I20260811 12:04:02.159631 25766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:04:02.160051 25766 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:04:02.165104 25766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "66daa7847dd64942aa7d736f874f75ea"
format_stamp: "Formatted at 2026-08-11 12:04:02 on dist-test-slave-mjjr"
server_key: "e376374b5b887225fd9c553ac8a1c611"
server_key_iv: "ab8a83bf30747b99e897e5f09e2b2931"
server_key_version: "encryptionkey@0"
I20260811 12:04:02.165886 25766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "66daa7847dd64942aa7d736f874f75ea"
format_stamp: "Formatted at 2026-08-11 12:04:02 on dist-test-slave-mjjr"
server_key: "e376374b5b887225fd9c553ac8a1c611"
server_key_iv: "ab8a83bf30747b99e897e5f09e2b2931"
server_key_version: "encryptionkey@0"
I20260811 12:04:02.170845 25766 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260811 12:04:02.174209 25785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:04:02.175521 25766 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:04:02.175756 25766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "66daa7847dd64942aa7d736f874f75ea"
format_stamp: "Formatted at 2026-08-11 12:04:02 on dist-test-slave-mjjr"
server_key: "e376374b5b887225fd9c553ac8a1c611"
server_key_iv: "ab8a83bf30747b99e897e5f09e2b2931"
server_key_version: "encryptionkey@0"
I20260811 12:04:02.175973 25766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:04:02.181479 25766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:04:02.182253 25766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:04:02.182559 25766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:04:02.200742 25766 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.190:43095
I20260811 12:04:02.200775 25854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.190:43095 every 8 connection(s)
I20260811 12:04:02.202069 25766 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:04:02.205717 25855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:04:02.208498 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 25766
I20260811 12:04:02.208699 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:04:02.209491 10102 external_mini_cluster.cc:1428] Setting key c95c1d6171a2580fd7b67f10e28bec3b
I20260811 12:04:02.217726 25861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:02.211603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33758 (local address 127.9.221.190:43095)
0811 12:04:02.212618 (+ 1015us) server_negotiation.cc:206] Beginning negotiation
0811 12:04:02.212628 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:04:02.212668 (+ 40us) server_negotiation.cc:407] Connection header received
0811 12:04:02.214276 (+ 1608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:04:02.214295 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:04:02.214690 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:04:02.215107 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:04:02.216229 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:04:02.216235 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:04:02.216246 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:04:02.216260 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:04:02.216313 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:04:02.216331 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:04:02.216505 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 12:04:02.216622 (+ 117us) server_negotiation.cc:299] Negotiation successful
0811 12:04:02.216878 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":373,"threads_started":1}
I20260811 12:04:02.218469 25855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66daa7847dd64942aa7d736f874f75ea: Bootstrap starting.
I20260811 12:04:02.221916 25855 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66daa7847dd64942aa7d736f874f75ea: Neither blocks nor log segments found. Creating new log.
I20260811 12:04:02.222918 25855 log.cc:824] T 00000000000000000000000000000000 P 66daa7847dd64942aa7d736f874f75ea: Log is configured to *not* fsync() on all Append() calls
I20260811 12:04:02.225605 25855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66daa7847dd64942aa7d736f874f75ea: No bootstrap required, opened a new log
I20260811 12:04:02.232780 25855 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66daa7847dd64942aa7d736f874f75ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66daa7847dd64942aa7d736f874f75ea" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 43095 } }
I20260811 12:04:02.233083 25855 raft_consensus.cc:374] T 00000000000000000000000000000000 P 66daa7847dd64942aa7d736f874f75ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:04:02.233155 25855 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66daa7847dd64942aa7d736f874f75ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66daa7847dd64942aa7d736f874f75ea, State: Initialized, Role: FOLLOWER
I20260811 12:04:02.233696 25855 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66daa7847dd64942aa7d736f874f75ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66daa7847dd64942aa7d736f874f75ea" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 43095 } }
I20260811 12:04:02.233893 25855 raft_consensus.cc:388] T 00000000000000000000000000000000 P 66daa7847dd64942aa7d736f874f75ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:04:02.233994 25855 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66daa7847dd64942aa7d736f874f75ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:04:02.234112 25855 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66daa7847dd64942aa7d736f874f75ea [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:04:02.236173 25855 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66daa7847dd64942aa7d736f874f75ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66daa7847dd64942aa7d736f874f75ea" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 43095 } }
I20260811 12:04:02.236697 25855 leader_election.cc:302] T 00000000000000000000000000000000 P 66daa7847dd64942aa7d736f874f75ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66daa7847dd64942aa7d736f874f75ea; no voters:
I20260811 12:04:02.237313 25855 leader_election.cc:288] T 00000000000000000000000000000000 P 66daa7847dd64942aa7d736f874f75ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:04:02.237550 25863 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66daa7847dd64942aa7d736f874f75ea [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:04:02.238273 25863 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66daa7847dd64942aa7d736f874f75ea [term 1 LEADER]: Becoming Leader. State: Replica: 66daa7847dd64942aa7d736f874f75ea, State: Running, Role: LEADER
I20260811 12:04:02.238824 25863 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66daa7847dd64942aa7d736f874f75ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66daa7847dd64942aa7d736f874f75ea" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 43095 } }
I20260811 12:04:02.239398 25855 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66daa7847dd64942aa7d736f874f75ea [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:04:02.241972 25864 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66daa7847dd64942aa7d736f874f75ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "66daa7847dd64942aa7d736f874f75ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66daa7847dd64942aa7d736f874f75ea" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 43095 } } }
I20260811 12:04:02.242408 25864 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66daa7847dd64942aa7d736f874f75ea [sys.catalog]: This master's current role is: LEADER
I20260811 12:04:02.242936 25865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66daa7847dd64942aa7d736f874f75ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66daa7847dd64942aa7d736f874f75ea. Latest consensus state: current_term: 1 leader_uuid: "66daa7847dd64942aa7d736f874f75ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66daa7847dd64942aa7d736f874f75ea" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 43095 } } }
I20260811 12:04:02.243350 25865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66daa7847dd64942aa7d736f874f75ea [sys.catalog]: This master's current role is: LEADER
I20260811 12:04:02.244608 25876 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:04:02.249774 25876 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:04:02.257804 25876 catalog_manager.cc:1347] Generated new cluster ID: 0c86fb4d881f486e96e1b980a4d25b7c
I20260811 12:04:02.257911 25876 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:04:02.277877 25876 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:04:02.278316 25876 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:04:02.290678 25876 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 66daa7847dd64942aa7d736f874f75ea: Generated new TSK 0
I20260811 12:04:02.291445 25876 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:04:02.296623 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.129:0
--local_ip_for_outbound_sockets=127.9.221.129
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:43095
--builtin_ntp_servers=127.9.221.148:41987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 12:04:02.634389 25888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:04:02.634661 25888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:04:02.634785 25888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:04:02.642467 25888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:04:02.642575 25888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:04:02.642725 25888 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.129
I20260811 12:04:02.652177 25888 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:41987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:43095
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.129
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:04:02.653563 25888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:04:02.655447 25888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:04:02.668934 25896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:04:02.669070 25895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:04:02.669044 25900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:04:02.670090 25888 server_base.cc:1034] running on GCE node
I20260811 12:04:02.670728 25888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:04:02.671795 25888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:04:02.673022 25888 hybrid_clock.cc:648] HybridClock initialized: now 1786449842672980 us; error 49 us; skew 500 ppm
I20260811 12:04:02.673341 25888 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:04:02.675485 25888 webserver.cc:460] Webserver started at http://127.9.221.129:43715/ using document root <none> and password file <none>
I20260811 12:04:02.676218 25888 fs_manager.cc:359] Metadata directory not provided
I20260811 12:04:02.676354 25888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:04:02.676677 25888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:04:02.679513 25888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/data/instance:
uuid: "ca7af4fc89534e53bdb1b709fdae1c46"
format_stamp: "Formatted at 2026-08-11 12:04:02 on dist-test-slave-mjjr"
server_key: "581a1b84ae1866e37d7a475339d6b1f7"
server_key_iv: "592456f83ee56d810010abd97cc1c210"
server_key_version: "encryptionkey@0"
I20260811 12:04:02.680451 25888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance:
uuid: "ca7af4fc89534e53bdb1b709fdae1c46"
format_stamp: "Formatted at 2026-08-11 12:04:02 on dist-test-slave-mjjr"
server_key: "581a1b84ae1866e37d7a475339d6b1f7"
server_key_iv: "592456f83ee56d810010abd97cc1c210"
server_key_version: "encryptionkey@0"
I20260811 12:04:02.685644 25888 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:04:02.688923 25906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:04:02.690312 25888 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260811 12:04:02.690474 25888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/wal
uuid: "ca7af4fc89534e53bdb1b709fdae1c46"
format_stamp: "Formatted at 2026-08-11 12:04:02 on dist-test-slave-mjjr"
server_key: "581a1b84ae1866e37d7a475339d6b1f7"
server_key_iv: "592456f83ee56d810010abd97cc1c210"
server_key_version: "encryptionkey@0"
I20260811 12:04:02.690685 25888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:04:02.696007 25888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:04:02.696744 25888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:04:02.697008 25888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:04:02.697841 25888 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:04:02.699180 25888 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:04:02.699291 25888 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:04:02.699412 25888 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:04:02.699514 25888 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:04:02.734752 25888 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.129:36787
I20260811 12:04:02.734804 26058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.129:36787 every 8 connection(s)
I20260811 12:04:02.736126 25888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
I20260811 12:04:02.743234 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 25888
I20260811 12:04:02.743448 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance
I20260811 12:04:02.744269 10102 external_mini_cluster.cc:1428] Setting key 723031ae84324cc957506d7913fc9bdd
I20260811 12:04:02.744525 26062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:02.738741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:39845 (local address 127.9.221.190:43095)
0811 12:04:02.739243 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0811 12:04:02.739248 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:04:02.740535 (+ 1287us) server_negotiation.cc:407] Connection header received
0811 12:04:02.742115 (+ 1580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:04:02.742119 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:04:02.742173 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:04:02.742269 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:04:02.743739 (+ 1470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:04:02.743743 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:04:02.743747 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:04:02.743763 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:04:02.743805 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:04:02.743816 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:04:02.743974 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 12:04:02.744147 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 12:04:02.744217 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":235,"threads_started":1}
I20260811 12:04:02.745568 26063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:02.739353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:43095 (local address 127.9.221.129:39845)
0811 12:04:02.740376 (+ 1023us) negotiation.cc:107] Waiting for socket to connect
0811 12:04:02.740416 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 12:04:02.741887 (+ 1471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:04:02.742465 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:04:02.742472 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:04:02.742554 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:04:02.743297 (+ 743us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:04:02.743300 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:04:02.743334 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:04:02.743466 (+ 132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:04:02.743917 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:04:02.743928 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:04:02.743936 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 12:04:02.744228 (+ 292us) client_negotiation.cc:240] Negotiation successful
0811 12:04:02.744468 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":330,"threads_started":1}
I20260811 12:04:02.747164 26060 heartbeater.cc:347] Connected to a master server at 127.9.221.190:43095
I20260811 12:04:02.747457 26060 heartbeater.cc:464] Registering TS with master...
I20260811 12:04:02.747579 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.130:0
--local_ip_for_outbound_sockets=127.9.221.130
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:43095
--builtin_ntp_servers=127.9.221.148:41987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 12:04:02.748173 26060 heartbeater.cc:511] Master 127.9.221.190:43095 requested a full tablet report, sending...
I20260811 12:04:02.750385 25807 ts_manager.cc:194] Registered new tserver with Master: ca7af4fc89534e53bdb1b709fdae1c46 (127.9.221.129:36787)
W20260811 12:04:03.087688 26064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:04:03.088039 26064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:04:03.088210 26064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:04:03.095535 26064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:04:03.095691 26064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:04:03.095875 26064 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.130
I20260811 12:04:03.104889 26064 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:41987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:43095
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.130
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:04:03.106215 26064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:04:03.108134 26064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:04:03.121601 26074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:04:03.121567 26071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:04:03.121672 26072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:04:03.124976 26064 server_base.cc:1034] running on GCE node
I20260811 12:04:03.125538 26064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:04:03.126449 26064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:04:03.127724 26064 hybrid_clock.cc:648] HybridClock initialized: now 1786449843127690 us; error 47 us; skew 500 ppm
I20260811 12:04:03.128055 26064 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:04:03.134565 26064 webserver.cc:460] Webserver started at http://127.9.221.130:40719/ using document root <none> and password file <none>
I20260811 12:04:03.135251 26064 fs_manager.cc:359] Metadata directory not provided
I20260811 12:04:03.135385 26064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:04:03.135762 26064 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:04:03.138616 26064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/data/instance:
uuid: "c938341ce00b4f5ca12b53df435f9c0f"
format_stamp: "Formatted at 2026-08-11 12:04:03 on dist-test-slave-mjjr"
server_key: "47533f4daa50ca2113a712e8a14d196e"
server_key_iv: "93a7ca9070ccc4cca49334405975d5ff"
server_key_version: "encryptionkey@0"
I20260811 12:04:03.139485 26064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance:
uuid: "c938341ce00b4f5ca12b53df435f9c0f"
format_stamp: "Formatted at 2026-08-11 12:04:03 on dist-test-slave-mjjr"
server_key: "47533f4daa50ca2113a712e8a14d196e"
server_key_iv: "93a7ca9070ccc4cca49334405975d5ff"
server_key_version: "encryptionkey@0"
I20260811 12:04:03.144608 26064 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 12:04:03.148033 26082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:04:03.149407 26064 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 12:04:03.149540 26064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/wal
uuid: "c938341ce00b4f5ca12b53df435f9c0f"
format_stamp: "Formatted at 2026-08-11 12:04:03 on dist-test-slave-mjjr"
server_key: "47533f4daa50ca2113a712e8a14d196e"
server_key_iv: "93a7ca9070ccc4cca49334405975d5ff"
server_key_version: "encryptionkey@0"
I20260811 12:04:03.149718 26064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:04:03.155171 26064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:04:03.155978 26064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:04:03.156199 26064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:04:03.157011 26064 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:04:03.158283 26064 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:04:03.158370 26064 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:04:03.158447 26064 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:04:03.158514 26064 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:04:03.192319 26064 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.130:43669
I20260811 12:04:03.192405 26246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.130:43669 every 8 connection(s)
I20260811 12:04:03.193679 26064 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
I20260811 12:04:03.195295 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 26064
I20260811 12:04:03.195499 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance
I20260811 12:04:03.196105 10102 external_mini_cluster.cc:1428] Setting key 6d791567807ae00b398d38c28b673344
I20260811 12:04:03.199159 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.131:0
--local_ip_for_outbound_sockets=127.9.221.131
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:43095
--builtin_ntp_servers=127.9.221.148:41987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 12:04:03.203158 26062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:03.196643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:57069 (local address 127.9.221.190:43095)
0811 12:04:03.196825 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 12:04:03.196830 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:04:03.198475 (+ 1645us) server_negotiation.cc:407] Connection header received
0811 12:04:03.200469 (+ 1994us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:04:03.200477 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:04:03.200569 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:04:03.200680 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:04:03.202259 (+ 1579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:04:03.202267 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:04:03.202274 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:04:03.202297 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:04:03.202345 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:04:03.202363 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:04:03.202493 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 12:04:03.202735 (+ 242us) server_negotiation.cc:299] Negotiation successful
0811 12:04:03.202801 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 12:04:03.204140 26249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:03.197229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:43095 (local address 127.9.221.130:57069)
0811 12:04:03.198315 (+ 1086us) negotiation.cc:107] Waiting for socket to connect
0811 12:04:03.198358 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 12:04:03.200165 (+ 1807us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:04:03.200946 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:04:03.200968 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:04:03.201095 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:04:03.201892 (+ 797us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:04:03.201895 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:04:03.201945 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:04:03.202062 (+ 117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:04:03.202485 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:04:03.202499 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:04:03.202509 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 12:04:03.202798 (+ 289us) client_negotiation.cc:240] Negotiation successful
0811 12:04:03.203033 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":299,"threads_started":1}
I20260811 12:04:03.205647 26247 heartbeater.cc:347] Connected to a master server at 127.9.221.190:43095
I20260811 12:04:03.205965 26247 heartbeater.cc:464] Registering TS with master...
I20260811 12:04:03.206641 26247 heartbeater.cc:511] Master 127.9.221.190:43095 requested a full tablet report, sending...
I20260811 12:04:03.208185 25807 ts_manager.cc:194] Registered new tserver with Master: c938341ce00b4f5ca12b53df435f9c0f (127.9.221.130:43669)
W20260811 12:04:03.529768 26250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:04:03.530040 26250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:04:03.530128 26250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:04:03.537143 26250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:04:03.537233 26250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:04:03.537457 26250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.131
I20260811 12:04:03.546496 26250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:41987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:43095
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.131
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:04:03.547766 26250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:04:03.549539 26250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:04:03.562947 26257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:04:03.563011 26261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:04:03.563028 26258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:04:03.564116 26250 server_base.cc:1034] running on GCE node
I20260811 12:04:03.564723 26250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:04:03.565580 26250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:04:03.566798 26250 hybrid_clock.cc:648] HybridClock initialized: now 1786449843566749 us; error 56 us; skew 500 ppm
I20260811 12:04:03.567094 26250 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:04:03.569393 26250 webserver.cc:460] Webserver started at http://127.9.221.131:41225/ using document root <none> and password file <none>
I20260811 12:04:03.570109 26250 fs_manager.cc:359] Metadata directory not provided
I20260811 12:04:03.570242 26250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:04:03.570577 26250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:04:03.573366 26250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/data/instance:
uuid: "229a3cf9ef5644a4b7e6ffa68aa0b973"
format_stamp: "Formatted at 2026-08-11 12:04:03 on dist-test-slave-mjjr"
server_key: "27e67facc1c680c8b5b30a8447265bab"
server_key_iv: "e9c967a6ff982937827becf14d4c563b"
server_key_version: "encryptionkey@0"
I20260811 12:04:03.574160 26250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance:
uuid: "229a3cf9ef5644a4b7e6ffa68aa0b973"
format_stamp: "Formatted at 2026-08-11 12:04:03 on dist-test-slave-mjjr"
server_key: "27e67facc1c680c8b5b30a8447265bab"
server_key_iv: "e9c967a6ff982937827becf14d4c563b"
server_key_version: "encryptionkey@0"
I20260811 12:04:03.579107 26250 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 12:04:03.582365 26268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:04:03.583607 26250 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 12:04:03.583846 26250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/wal
uuid: "229a3cf9ef5644a4b7e6ffa68aa0b973"
format_stamp: "Formatted at 2026-08-11 12:04:03 on dist-test-slave-mjjr"
server_key: "27e67facc1c680c8b5b30a8447265bab"
server_key_iv: "e9c967a6ff982937827becf14d4c563b"
server_key_version: "encryptionkey@0"
I20260811 12:04:03.584071 26250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:04:03.589478 26250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:04:03.590216 26250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:04:03.590499 26250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:04:03.591358 26250 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:04:03.592787 26250 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:04:03.592912 26250 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:04:03.593040 26250 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:04:03.593127 26250 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:04:03.626137 26250 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.131:45671
I20260811 12:04:03.626219 26433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.131:45671 every 8 connection(s)
I20260811 12:04:03.627413 26250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
I20260811 12:04:03.635015 26062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:03.630001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:47461 (local address 127.9.221.190:43095)
0811 12:04:03.630175 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 12:04:03.630180 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:04:03.631608 (+ 1428us) server_negotiation.cc:407] Connection header received
0811 12:04:03.633143 (+ 1535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:04:03.633147 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:04:03.633200 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:04:03.633288 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:04:03.634430 (+ 1142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:04:03.634434 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:04:03.634437 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:04:03.634452 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:04:03.634483 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:04:03.634494 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:04:03.634592 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 12:04:03.634738 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 12:04:03.634786 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 12:04:03.635936 26437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:03.630497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:43095 (local address 127.9.221.131:47461)
0811 12:04:03.631367 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0811 12:04:03.631411 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 12:04:03.632892 (+ 1481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:04:03.633474 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:04:03.633481 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:04:03.633559 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:04:03.634170 (+ 611us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:04:03.634172 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:04:03.634206 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:04:03.634281 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:04:03.634575 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:04:03.634582 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:04:03.634587 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 12:04:03.634781 (+ 194us) client_negotiation.cc:240] Negotiation successful
0811 12:04:03.634978 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":262,"threads_started":1}
I20260811 12:04:03.636561 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 26250
I20260811 12:04:03.636751 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance
I20260811 12:04:03.637341 10102 external_mini_cluster.cc:1428] Setting key 0dcc5586ebecaae29f9920ae6d0c7181
I20260811 12:04:03.637385 26434 heartbeater.cc:347] Connected to a master server at 127.9.221.190:43095
I20260811 12:04:03.637673 26434 heartbeater.cc:464] Registering TS with master...
I20260811 12:04:03.638314 26434 heartbeater.cc:511] Master 127.9.221.190:43095 requested a full tablet report, sending...
I20260811 12:04:03.640010 25804 ts_manager.cc:194] Registered new tserver with Master: 229a3cf9ef5644a4b7e6ffa68aa0b973 (127.9.221.131:45671)
I20260811 12:04:03.652793 10102 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 12:04:03.659009 26062 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:04:03.657329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33762 (local address 127.9.221.190:43095)
0811 12:04:03.657492 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 12:04:03.657496 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:04:03.657993 (+ 497us) server_negotiation.cc:407] Connection header received
0811 12:04:03.658183 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:04:03.658189 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:04:03.658243 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:04:03.658343 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:04:03.658739 (+ 396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33762: BlockingRecv error: recv got EOF from 127.0.0.1:33762 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 12:04:03.659459 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 25888
I20260811 12:04:03.662845 26447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:03.660574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52514 (local address 127.9.221.129:36787)
0811 12:04:03.661084 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0811 12:04:03.661091 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:04:03.661114 (+ 23us) server_negotiation.cc:407] Connection header received
0811 12:04:03.661169 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:04:03.661174 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:04:03.661346 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:04:03.661465 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:04:03.662273 (+ 808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:04:03.662278 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:04:03.662287 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:04:03.662303 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:04:03.662339 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:04:03.662352 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:04:03.662458 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 12:04:03.662539 (+ 81us) server_negotiation.cc:299] Negotiation successful
0811 12:04:03.662609 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":253,"threads_started":1}
I20260811 12:04:03.663990 26054 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:04:03.745443 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 25888
I20260811 12:04:03.771481 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 26064
I20260811 12:04:03.774953 26449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:03.772397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49332 (local address 127.9.221.130:43669)
0811 12:04:03.772970 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0811 12:04:03.772976 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:04:03.773005 (+ 29us) server_negotiation.cc:407] Connection header received
0811 12:04:03.773071 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:04:03.773077 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:04:03.773282 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:04:03.773410 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:04:03.774322 (+ 912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:04:03.774328 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:04:03.774351 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:04:03.774368 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:04:03.774404 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:04:03.774417 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:04:03.774537 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 12:04:03.774634 (+ 97us) server_negotiation.cc:299] Negotiation successful
0811 12:04:03.774714 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":270,"threads_started":1}
I20260811 12:04:03.776288 26242 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:04:03.858289 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 26064
I20260811 12:04:03.885044 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 26250
I20260811 12:04:03.889029 26452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:03.886127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37044 (local address 127.9.221.131:45671)
0811 12:04:03.886747 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0811 12:04:03.886754 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:04:03.886784 (+ 30us) server_negotiation.cc:407] Connection header received
0811 12:04:03.886847 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:04:03.886853 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:04:03.887086 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:04:03.887216 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:04:03.888309 (+ 1093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:04:03.888315 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:04:03.888326 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:04:03.888342 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:04:03.888380 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:04:03.888393 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:04:03.888599 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0811 12:04:03.888658 (+ 59us) server_negotiation.cc:299] Negotiation successful
0811 12:04:03.888739 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":299,"threads_started":1}
I20260811 12:04:03.890185 26426 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:04:03.971021 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 26250
I20260811 12:04:03.993749 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 25766
I20260811 12:04:03.995007 25850 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:04:04.097450 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 25766
2026-08-11T12:04:04Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2400 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:04:04 dist-test-slave-mjjr krb5kdc[26461](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:04:04 dist-test-slave-mjjr krb5kdc[26461](info): set up 2 sockets
Aug 11 12:04:04 dist-test-slave-mjjr krb5kdc[26461](info): commencing operation
krb5kdc: starting...
W20260811 12:04:05.449369 10102 mini_kdc.cc:121] Time spent starting KDC: real 1.300s user 0.006s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Aug 11 12:04:05 dist-test-slave-mjjr krb5kdc[26461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449845, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:04:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:04:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.221.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
I20260811 12:04:05.710594 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:42107
--webserver_interface=127.9.221.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:43603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:42107
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:04:06.041049 26485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:04:06.041422 26485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:04:06.041572 26485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:04:06.048906 26485 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:04:06.049041 26485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:04:06.049149 26485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:04:06.049249 26485 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:04:06.049322 26485 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:04:06.059382 26485 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:43603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:42107
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:42107
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.221.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:04:06.060792 26485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:04:06.062124 26485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:04:06.074381 26493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:04:06.074397 26497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:04:06.074463 26495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:04:06.081243 26485 server_base.cc:1034] running on GCE node
I20260811 12:04:06.082041 26485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:04:06.083434 26485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:04:06.084676 26485 hybrid_clock.cc:648] HybridClock initialized: now 1786449846084631 us; error 38 us; skew 500 ppm
I20260811 12:04:06.085040 26485 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:04:06 dist-test-slave-mjjr krb5kdc[26461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449846, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:04:06.088207 26485 init.cc:377] Logged in from keytab as kudu/127.9.221.190@KRBTEST.COM (short username kudu)
I20260811 12:04:06.089751 26485 webserver.cc:460] Webserver started at http://127.9.221.190:46551/ using document root <none> and password file <none>
I20260811 12:04:06.090456 26485 fs_manager.cc:359] Metadata directory not provided
I20260811 12:04:06.090603 26485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:04:06.090919 26485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:04:06.093899 26485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "d11e9824bd6f4eb7b9f04021343ea734"
format_stamp: "Formatted at 2026-08-11 12:04:06 on dist-test-slave-mjjr"
server_key: "4084f5a5f7438af5b50ecd7f7d2d8abf"
server_key_iv: "fe75529c76f3234469ac16d3348722cc"
server_key_version: "encryptionkey@0"
I20260811 12:04:06.094755 26485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "d11e9824bd6f4eb7b9f04021343ea734"
format_stamp: "Formatted at 2026-08-11 12:04:06 on dist-test-slave-mjjr"
server_key: "4084f5a5f7438af5b50ecd7f7d2d8abf"
server_key_iv: "fe75529c76f3234469ac16d3348722cc"
server_key_version: "encryptionkey@0"
I20260811 12:04:06.099833 26485 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260811 12:04:06.103051 26507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:04:06.104434 26485 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:04:06.104614 26485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "d11e9824bd6f4eb7b9f04021343ea734"
format_stamp: "Formatted at 2026-08-11 12:04:06 on dist-test-slave-mjjr"
server_key: "4084f5a5f7438af5b50ecd7f7d2d8abf"
server_key_iv: "fe75529c76f3234469ac16d3348722cc"
server_key_version: "encryptionkey@0"
I20260811 12:04:06.104803 26485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:04:06.148095 26485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:04:06.152992 26485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:04:06.153301 26485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:04:06.172333 26485 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.190:42107
I20260811 12:04:06.172379 26588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.190:42107 every 8 connection(s)
I20260811 12:04:06.173695 26485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:04:06.175745 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 26485
I20260811 12:04:06.175979 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:04:06.176852 10102 external_mini_cluster.cc:1428] Setting key 6aaedf8fdd69a0df9f24e7555707a095
I20260811 12:04:06.178237 26589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 12:04:06 dist-test-slave-mjjr krb5kdc[26461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449845, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:04:06.192369 26589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d11e9824bd6f4eb7b9f04021343ea734: Bootstrap starting.
I20260811 12:04:06.195343 26589 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d11e9824bd6f4eb7b9f04021343ea734: Neither blocks nor log segments found. Creating new log.
I20260811 12:04:06.196724 26589 log.cc:824] T 00000000000000000000000000000000 P d11e9824bd6f4eb7b9f04021343ea734: Log is configured to *not* fsync() on all Append() calls
I20260811 12:04:06.199503 26592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:06.178938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42320 (local address 127.9.221.190:42107)
0811 12:04:06.179764 (+ 826us) server_negotiation.cc:206] Beginning negotiation
0811 12:04:06.179775 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:04:06.179815 (+ 40us) server_negotiation.cc:407] Connection header received
0811 12:04:06.181288 (+ 1473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:04:06.181307 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:04:06.181660 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:04:06.181956 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:04:06.183871 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:06.185148 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:06.186965 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:06.187590 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:06.187789 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:06.192402 (+ 4613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:04:06.192409 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:04:06.192420 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:04:06.192452 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:04:06.195405 (+ 2953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:04:06.196264 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:04:06.196270 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:04:06.196275 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:04:06.196438 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:04:06.196884 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:04:06.196893 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:04:06.196896 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:04:06.197531 (+ 635us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:04:06.197898 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0811 12:04:06.198193 (+ 295us) server_negotiation.cc:299] Negotiation successful
0811 12:04:06.198586 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":388,"threads_started":1}
I20260811 12:04:06.199751 26589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d11e9824bd6f4eb7b9f04021343ea734: No bootstrap required, opened a new log
I20260811 12:04:06.206825 26589 raft_consensus.cc:348] T 00000000000000000000000000000000 P d11e9824bd6f4eb7b9f04021343ea734 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d11e9824bd6f4eb7b9f04021343ea734" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 42107 } }
I20260811 12:04:06.207171 26589 raft_consensus.cc:374] T 00000000000000000000000000000000 P d11e9824bd6f4eb7b9f04021343ea734 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:04:06.207294 26589 raft_consensus.cc:729] T 00000000000000000000000000000000 P d11e9824bd6f4eb7b9f04021343ea734 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d11e9824bd6f4eb7b9f04021343ea734, State: Initialized, Role: FOLLOWER
I20260811 12:04:06.207929 26589 consensus_queue.cc:260] T 00000000000000000000000000000000 P d11e9824bd6f4eb7b9f04021343ea734 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d11e9824bd6f4eb7b9f04021343ea734" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 42107 } }
I20260811 12:04:06.208206 26589 raft_consensus.cc:388] T 00000000000000000000000000000000 P d11e9824bd6f4eb7b9f04021343ea734 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:04:06.208374 26589 raft_consensus.cc:482] T 00000000000000000000000000000000 P d11e9824bd6f4eb7b9f04021343ea734 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:04:06.208564 26589 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d11e9824bd6f4eb7b9f04021343ea734 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:04:06.210788 26589 raft_consensus.cc:504] T 00000000000000000000000000000000 P d11e9824bd6f4eb7b9f04021343ea734 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d11e9824bd6f4eb7b9f04021343ea734" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 42107 } }
I20260811 12:04:06.211265 26589 leader_election.cc:302] T 00000000000000000000000000000000 P d11e9824bd6f4eb7b9f04021343ea734 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d11e9824bd6f4eb7b9f04021343ea734; no voters:
I20260811 12:04:06.211875 26589 leader_election.cc:288] T 00000000000000000000000000000000 P d11e9824bd6f4eb7b9f04021343ea734 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:04:06.212003 26595 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d11e9824bd6f4eb7b9f04021343ea734 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:04:06.212724 26595 raft_consensus.cc:686] T 00000000000000000000000000000000 P d11e9824bd6f4eb7b9f04021343ea734 [term 1 LEADER]: Becoming Leader. State: Replica: d11e9824bd6f4eb7b9f04021343ea734, State: Running, Role: LEADER
I20260811 12:04:06.213320 26595 consensus_queue.cc:237] T 00000000000000000000000000000000 P d11e9824bd6f4eb7b9f04021343ea734 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d11e9824bd6f4eb7b9f04021343ea734" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 42107 } }
I20260811 12:04:06.214424 26589 sys_catalog.cc:564] T 00000000000000000000000000000000 P d11e9824bd6f4eb7b9f04021343ea734 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:04:06.216109 26597 sys_catalog.cc:455] T 00000000000000000000000000000000 P d11e9824bd6f4eb7b9f04021343ea734 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d11e9824bd6f4eb7b9f04021343ea734" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d11e9824bd6f4eb7b9f04021343ea734" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 42107 } } }
I20260811 12:04:06.216511 26597 sys_catalog.cc:458] T 00000000000000000000000000000000 P d11e9824bd6f4eb7b9f04021343ea734 [sys.catalog]: This master's current role is: LEADER
I20260811 12:04:06.218648 26605 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:04:06.218449 26596 sys_catalog.cc:455] T 00000000000000000000000000000000 P d11e9824bd6f4eb7b9f04021343ea734 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d11e9824bd6f4eb7b9f04021343ea734. Latest consensus state: current_term: 1 leader_uuid: "d11e9824bd6f4eb7b9f04021343ea734" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d11e9824bd6f4eb7b9f04021343ea734" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 42107 } } }
I20260811 12:04:06.218837 26596 sys_catalog.cc:458] T 00000000000000000000000000000000 P d11e9824bd6f4eb7b9f04021343ea734 [sys.catalog]: This master's current role is: LEADER
I20260811 12:04:06.223898 26605 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:04:06.233117 26605 catalog_manager.cc:1347] Generated new cluster ID: b88faadee633458bb0b9dd0107cc9a8a
I20260811 12:04:06.233237 26605 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:04:06.258461 26605 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:04:06.260298 26605 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:04:06.272727 26605 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d11e9824bd6f4eb7b9f04021343ea734: Generated new TSK 0
I20260811 12:04:06.273545 26605 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.221.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab.
Entry for principal kudu/127.9.221.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab.
I20260811 12:04:06.348868 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.129:0
--local_ip_for_outbound_sockets=127.9.221.129
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:42107
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab
--principal=kudu/127.9.221.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:43603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:04:06.684278 26628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:04:06.684530 26628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:04:06.684733 26628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:04:06.691844 26628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:04:06.691974 26628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:04:06.692173 26628 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.129
I20260811 12:04:06.702211 26628 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:43603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab
--principal=kudu/127.9.221.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:42107
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.129
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:04:06.703513 26628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:04:06.704842 26628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:04:06.717975 26634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:04:06.718379 26638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:04:06.718504 26633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:04:06.718966 26628 server_base.cc:1034] running on GCE node
I20260811 12:04:06.719609 26628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:04:06.720534 26628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:04:06.721771 26628 hybrid_clock.cc:648] HybridClock initialized: now 1786449846721711 us; error 43 us; skew 500 ppm
I20260811 12:04:06.722075 26628 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:04:06 dist-test-slave-mjjr krb5kdc[26461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449846, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:04:06.725643 26628 init.cc:377] Logged in from keytab as kudu/127.9.221.129@KRBTEST.COM (short username kudu)
I20260811 12:04:06.727608 26628 webserver.cc:460] Webserver started at http://127.9.221.129:43253/ using document root <none> and password file <none>
I20260811 12:04:06.728355 26628 fs_manager.cc:359] Metadata directory not provided
I20260811 12:04:06.728494 26628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:04:06.728806 26628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:04:06.731714 26628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/data/instance:
uuid: "a9a70fd2831c40269b8c296dd3a3b771"
format_stamp: "Formatted at 2026-08-11 12:04:06 on dist-test-slave-mjjr"
server_key: "ff5a78ee1c402c68a04bc9fed845fd52"
server_key_iv: "3a8e2cf13c746da0d2217f26a20eee28"
server_key_version: "encryptionkey@0"
I20260811 12:04:06.732488 26628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance:
uuid: "a9a70fd2831c40269b8c296dd3a3b771"
format_stamp: "Formatted at 2026-08-11 12:04:06 on dist-test-slave-mjjr"
server_key: "ff5a78ee1c402c68a04bc9fed845fd52"
server_key_iv: "3a8e2cf13c746da0d2217f26a20eee28"
server_key_version: "encryptionkey@0"
I20260811 12:04:06.737545 26628 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 12:04:06.740803 26650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:04:06.742074 26628 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:04:06.742259 26628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/wal
uuid: "a9a70fd2831c40269b8c296dd3a3b771"
format_stamp: "Formatted at 2026-08-11 12:04:06 on dist-test-slave-mjjr"
server_key: "ff5a78ee1c402c68a04bc9fed845fd52"
server_key_iv: "3a8e2cf13c746da0d2217f26a20eee28"
server_key_version: "encryptionkey@0"
I20260811 12:04:06.742444 26628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:04:06.755848 26628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:04:06.759586 26628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:04:06.759943 26628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:04:06.760816 26628 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:04:06.762310 26628 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:04:06.762398 26628 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:04:06.762588 26628 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:04:06.762679 26628 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:04:06.799494 26628 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.129:37149
I20260811 12:04:06.799532 26815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.129:37149 every 8 connection(s)
I20260811 12:04:06.800930 26628 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
I20260811 12:04:06.804749 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 26628
I20260811 12:04:06.804986 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance
I20260811 12:04:06.805832 10102 external_mini_cluster.cc:1428] Setting key d57052c4366a06428a61e3d4f26fd778
Aug 11 12:04:06 dist-test-slave-mjjr krb5kdc[26461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449846, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:04:06.823349 26819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:06.803527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:44037 (local address 127.9.221.190:42107)
0811 12:04:06.804126 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0811 12:04:06.804133 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:04:06.805209 (+ 1076us) server_negotiation.cc:407] Connection header received
0811 12:04:06.807404 (+ 2195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:04:06.807410 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:04:06.807500 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:04:06.807616 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:04:06.809604 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:06.810336 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:06.812754 (+ 2418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:06.813157 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:06.813259 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:06.817480 (+ 4221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:04:06.817484 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:04:06.817488 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:04:06.817519 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:04:06.820293 (+ 2774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:04:06.821030 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:04:06.821034 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:04:06.821036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:04:06.821153 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:04:06.821615 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:04:06.821618 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:04:06.821620 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:04:06.821874 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:04:06.822039 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 12:04:06.822731 (+ 692us) server_negotiation.cc:299] Negotiation successful
0811 12:04:06.822984 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":280,"threads_started":1}
I20260811 12:04:06.824324 26822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:06.804162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:42107 (local address 127.9.221.129:44037)
0811 12:04:06.805042 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0811 12:04:06.805090 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 12:04:06.807148 (+ 2058us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:04:06.807895 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:04:06.807906 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:04:06.808619 (+ 713us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:04:06.809413 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:04:06.809449 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:04:06.810506 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:04:06.810514 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:04:06.812608 (+ 2094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:04:06.812625 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:04:06.813380 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:04:06.813390 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:04:06.813720 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:06.814492 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:04:06.814523 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:04:06.817246 (+ 2723us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:04:06.820449 (+ 3203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:04:06.820466 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:04:06.820481 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:04:06.820883 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:04:06.821268 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:04:06.821273 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:04:06.821277 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:04:06.821485 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:04:06.822024 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:04:06.822031 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:04:06.822352 (+ 321us) client_negotiation.cc:769] Sending connection context
0811 12:04:06.822612 (+ 260us) client_negotiation.cc:240] Negotiation successful
0811 12:04:06.823138 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":320,"threads_started":1}
I20260811 12:04:06.825868 26817 heartbeater.cc:347] Connected to a master server at 127.9.221.190:42107
I20260811 12:04:06.826216 26817 heartbeater.cc:464] Registering TS with master...
I20260811 12:04:06.826961 26817 heartbeater.cc:511] Master 127.9.221.190:42107 requested a full tablet report, sending...
I20260811 12:04:06.829286 26533 ts_manager.cc:194] Registered new tserver with Master: a9a70fd2831c40269b8c296dd3a3b771 (127.9.221.129:37149)
I20260811 12:04:06.831516 26533 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.129@KRBTEST.COM'} at 127.9.221.129:44037
WARNING: no policy specified for kudu/127.9.221.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab.
Entry for principal kudu/127.9.221.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab.
I20260811 12:04:06.864526 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.130:0
--local_ip_for_outbound_sockets=127.9.221.130
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:42107
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab
--principal=kudu/127.9.221.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:43603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:04:07.203022 26826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:04:07.203279 26826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:04:07.203457 26826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:04:07.210593 26826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:04:07.210709 26826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:04:07.210893 26826 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.130
I20260811 12:04:07.220409 26826 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:43603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab
--principal=kudu/127.9.221.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:42107
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.130
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:04:07.221710 26826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:04:07.222988 26826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:04:07.236117 26841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:04:07.236207 26835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:04:07.236164 26836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:04:07.237066 26826 server_base.cc:1034] running on GCE node
I20260811 12:04:07.237635 26826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:04:07.238577 26826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:04:07.239791 26826 hybrid_clock.cc:648] HybridClock initialized: now 1786449847239733 us; error 47 us; skew 500 ppm
I20260811 12:04:07.240104 26826 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:04:07 dist-test-slave-mjjr krb5kdc[26461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449847, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:04:07.243173 26826 init.cc:377] Logged in from keytab as kudu/127.9.221.130@KRBTEST.COM (short username kudu)
I20260811 12:04:07.244544 26826 webserver.cc:460] Webserver started at http://127.9.221.130:44643/ using document root <none> and password file <none>
I20260811 12:04:07.245201 26826 fs_manager.cc:359] Metadata directory not provided
I20260811 12:04:07.245293 26826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:04:07.245581 26826 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:04:07.248785 26826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/data/instance:
uuid: "a31eff20048d47099f7183ff9c0990c8"
format_stamp: "Formatted at 2026-08-11 12:04:07 on dist-test-slave-mjjr"
server_key: "654efc28de7e133f3e3e8729436ace04"
server_key_iv: "7256facec64613c1d34f244a7ff2a991"
server_key_version: "encryptionkey@0"
I20260811 12:04:07.249600 26826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance:
uuid: "a31eff20048d47099f7183ff9c0990c8"
format_stamp: "Formatted at 2026-08-11 12:04:07 on dist-test-slave-mjjr"
server_key: "654efc28de7e133f3e3e8729436ace04"
server_key_iv: "7256facec64613c1d34f244a7ff2a991"
server_key_version: "encryptionkey@0"
I20260811 12:04:07.254629 26826 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 12:04:07.257975 26851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:04:07.259353 26826 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 12:04:07.259533 26826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/wal
uuid: "a31eff20048d47099f7183ff9c0990c8"
format_stamp: "Formatted at 2026-08-11 12:04:07 on dist-test-slave-mjjr"
server_key: "654efc28de7e133f3e3e8729436ace04"
server_key_iv: "7256facec64613c1d34f244a7ff2a991"
server_key_version: "encryptionkey@0"
I20260811 12:04:07.259785 26826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:04:07.286573 26826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:04:07.290762 26826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:04:07.291086 26826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:04:07.292140 26826 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:04:07.293488 26826 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:04:07.293565 26826 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:04:07.293746 26826 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:04:07.293799 26826 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:04:07.328275 26826 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.130:37087
I20260811 12:04:07.328402 26997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.130:37087 every 8 connection(s)
I20260811 12:04:07.329586 26826 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
I20260811 12:04:07.330246 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 26826
I20260811 12:04:07.330467 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance
I20260811 12:04:07.330972 10102 external_mini_cluster.cc:1428] Setting key 4f64d602f45439151414ad036940e42e
Aug 11 12:04:07 dist-test-slave-mjjr krb5kdc[26461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449847, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.130@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:04:07.350718 27000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:07.332402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:58561 (local address 127.9.221.190:42107)
0811 12:04:07.332876 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0811 12:04:07.332882 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:04:07.334000 (+ 1118us) server_negotiation.cc:407] Connection header received
0811 12:04:07.335849 (+ 1849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:04:07.335855 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:04:07.335944 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:04:07.336063 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:04:07.337509 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:07.338237 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:07.339822 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:07.340268 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:07.340405 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:07.344213 (+ 3808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:04:07.344222 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:04:07.344229 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:04:07.344276 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:04:07.347389 (+ 3113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:04:07.348363 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:04:07.348370 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:04:07.348375 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:04:07.348517 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:04:07.349019 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:04:07.349022 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:04:07.349024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:04:07.349274 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:04:07.349381 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 12:04:07.350254 (+ 873us) server_negotiation.cc:299] Negotiation successful
0811 12:04:07.350419 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":222,"threads_started":1}
I20260811 12:04:07.352296 27004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:07.333075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:42107 (local address 127.9.221.130:58561)
0811 12:04:07.333842 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0811 12:04:07.333884 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 12:04:07.335545 (+ 1661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:04:07.336283 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:04:07.336290 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:04:07.336758 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:04:07.337337 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:04:07.337354 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:04:07.338395 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:04:07.338399 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:04:07.339628 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:04:07.339697 (+ 69us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:04:07.340522 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:04:07.340529 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:04:07.340763 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:07.341454 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:04:07.341477 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:04:07.343957 (+ 2480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:04:07.347533 (+ 3576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:04:07.347545 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:04:07.347563 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:04:07.348220 (+ 657us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:04:07.348628 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:04:07.348636 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:04:07.348642 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:04:07.348897 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:04:07.349403 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:04:07.349415 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:04:07.349859 (+ 444us) client_negotiation.cc:769] Sending connection context
0811 12:04:07.350162 (+ 303us) client_negotiation.cc:240] Negotiation successful
0811 12:04:07.350778 (+ 616us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":328,"threads_started":1}
I20260811 12:04:07.353776 26998 heartbeater.cc:347] Connected to a master server at 127.9.221.190:42107
I20260811 12:04:07.354101 26998 heartbeater.cc:464] Registering TS with master...
I20260811 12:04:07.354964 26998 heartbeater.cc:511] Master 127.9.221.190:42107 requested a full tablet report, sending...
I20260811 12:04:07.356578 26533 ts_manager.cc:194] Registered new tserver with Master: a31eff20048d47099f7183ff9c0990c8 (127.9.221.130:37087)
I20260811 12:04:07.357568 26533 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.130@KRBTEST.COM'} at 127.9.221.130:58561
WARNING: no policy specified for kudu/127.9.221.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab.
Entry for principal kudu/127.9.221.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab.
I20260811 12:04:07.394033 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.131:0
--local_ip_for_outbound_sockets=127.9.221.131
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:42107
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab
--principal=kudu/127.9.221.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:43603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:04:07.743580 27010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:04:07.743906 27010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:04:07.744063 27010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:04:07.751384 27010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:04:07.751535 27010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:04:07.751778 27010 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.131
I20260811 12:04:07.761806 27010 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:43603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab
--principal=kudu/127.9.221.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:42107
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.131
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:04:07.763091 27010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:04:07.764603 27010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:04:07.777453 27016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:04:07.777488 27017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:04:07.777427 27020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:04:07.778206 27010 server_base.cc:1034] running on GCE node
I20260811 12:04:07.778855 27010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:04:07.779800 27010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:04:07.781032 27010 hybrid_clock.cc:648] HybridClock initialized: now 1786449847780965 us; error 54 us; skew 500 ppm
I20260811 12:04:07.781358 27010 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:04:07 dist-test-slave-mjjr krb5kdc[26461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449847, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:04:07.784437 27010 init.cc:377] Logged in from keytab as kudu/127.9.221.131@KRBTEST.COM (short username kudu)
I20260811 12:04:07.785859 27010 webserver.cc:460] Webserver started at http://127.9.221.131:46497/ using document root <none> and password file <none>
I20260811 12:04:07.786571 27010 fs_manager.cc:359] Metadata directory not provided
I20260811 12:04:07.786715 27010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:04:07.787027 27010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:04:07.789954 27010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/data/instance:
uuid: "0299f45f4e9e4affaff259752d1228f5"
format_stamp: "Formatted at 2026-08-11 12:04:07 on dist-test-slave-mjjr"
server_key: "1d207f94659aa54d7a49ed5385459831"
server_key_iv: "1e524d1c7b1c24b9dbfb4bb8d1d35df1"
server_key_version: "encryptionkey@0"
I20260811 12:04:07.790704 27010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance:
uuid: "0299f45f4e9e4affaff259752d1228f5"
format_stamp: "Formatted at 2026-08-11 12:04:07 on dist-test-slave-mjjr"
server_key: "1d207f94659aa54d7a49ed5385459831"
server_key_iv: "1e524d1c7b1c24b9dbfb4bb8d1d35df1"
server_key_version: "encryptionkey@0"
I20260811 12:04:07.795742 27010 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260811 12:04:07.799162 27033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:04:07.800653 27010 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 12:04:07.800807 27010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/wal
uuid: "0299f45f4e9e4affaff259752d1228f5"
format_stamp: "Formatted at 2026-08-11 12:04:07 on dist-test-slave-mjjr"
server_key: "1d207f94659aa54d7a49ed5385459831"
server_key_iv: "1e524d1c7b1c24b9dbfb4bb8d1d35df1"
server_key_version: "encryptionkey@0"
I20260811 12:04:07.800994 27010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:04:07.819761 27010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:04:07.823448 27010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:04:07.823817 27010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:04:07.824753 27010 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:04:07.826074 27010 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:04:07.826146 27010 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:04:07.826318 27010 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:04:07.826368 27010 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:04:07.834472 26817 heartbeater.cc:503] Master 127.9.221.190:42107 was elected leader, sending a full tablet report...
I20260811 12:04:07.861119 27010 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.131:36839
I20260811 12:04:07.861158 27191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.131:36839 every 8 connection(s)
I20260811 12:04:07.862378 27010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
I20260811 12:04:07.869918 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 27010
I20260811 12:04:07.870097 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance
I20260811 12:04:07.870688 10102 external_mini_cluster.cc:1428] Setting key 370a55be4fb08f675063c779af6fb21b
Aug 11 12:04:07 dist-test-slave-mjjr krb5kdc[26461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449847, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.131@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:04:07.885305 27195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:07.865376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:41155 (local address 127.9.221.190:42107)
0811 12:04:07.866044 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0811 12:04:07.866057 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 12:04:07.867294 (+ 1237us) server_negotiation.cc:407] Connection header received
0811 12:04:07.870239 (+ 2945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:04:07.870247 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:04:07.870355 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:04:07.870479 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:04:07.872923 (+ 2444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:07.874233 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:07.875874 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:07.876300 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:07.876435 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:07.879727 (+ 3292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:04:07.879733 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:04:07.879740 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:04:07.879779 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:04:07.882152 (+ 2373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:04:07.882869 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:04:07.882874 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:04:07.882878 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:04:07.883018 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:04:07.883464 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:04:07.883470 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:04:07.883473 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:04:07.883823 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:04:07.884019 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 12:04:07.884677 (+ 658us) server_negotiation.cc:299] Negotiation successful
0811 12:04:07.884904 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":338,"threads_started":1}
I20260811 12:04:07.886258 27196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:07.865992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:42107 (local address 127.9.221.131:41155)
0811 12:04:07.867084 (+ 1092us) negotiation.cc:107] Waiting for socket to connect
0811 12:04:07.867149 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 12:04:07.869798 (+ 2649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:04:07.870934 (+ 1136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:04:07.870944 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:04:07.871813 (+ 869us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:04:07.872676 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:04:07.872704 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:04:07.874392 (+ 1688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:04:07.874397 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:04:07.875721 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:04:07.875736 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:04:07.876556 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:04:07.876564 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:04:07.876785 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:07.877368 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:04:07.877397 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:04:07.879491 (+ 2094us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:04:07.882291 (+ 2800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:04:07.882300 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:04:07.882315 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:04:07.882733 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:04:07.883134 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:04:07.883144 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:04:07.883148 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:04:07.883337 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:04:07.883965 (+ 628us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:04:07.883972 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:04:07.884298 (+ 326us) client_negotiation.cc:769] Sending connection context
0811 12:04:07.884568 (+ 270us) client_negotiation.cc:240] Negotiation successful
0811 12:04:07.885084 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":750,"thread_start_us":412,"threads_started":1}
I20260811 12:04:07.887532 27192 heartbeater.cc:347] Connected to a master server at 127.9.221.190:42107
I20260811 12:04:07.887851 27192 heartbeater.cc:464] Registering TS with master...
I20260811 12:04:07.888578 27192 heartbeater.cc:511] Master 127.9.221.190:42107 requested a full tablet report, sending...
I20260811 12:04:07.889963 26533 ts_manager.cc:194] Registered new tserver with Master: 0299f45f4e9e4affaff259752d1228f5 (127.9.221.131:36839)
I20260811 12:04:07.890774 26533 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.131@KRBTEST.COM'} at 127.9.221.131:41155
I20260811 12:04:07.898840 10102 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:04:07.915215 27195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:07.903313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42328 (local address 127.9.221.190:42107)
0811 12:04:07.903464 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 12:04:07.903468 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:04:07.903984 (+ 516us) server_negotiation.cc:407] Connection header received
0811 12:04:07.904114 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:04:07.904117 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:04:07.904186 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:04:07.904262 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:04:07.905714 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:07.906393 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:07.907580 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:07.908037 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:07.908122 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:07.910491 (+ 2369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:04:07.910495 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:04:07.910498 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:04:07.910524 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:04:07.912862 (+ 2338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:04:07.913465 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:04:07.913468 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:04:07.913470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:04:07.913576 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:04:07.913880 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:04:07.913883 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:04:07.913885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:04:07.914182 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:04:07.914359 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 12:04:07.914789 (+ 430us) server_negotiation.cc:299] Negotiation successful
0811 12:04:07.914954 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 12:04:07.919024 26533 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42328:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:04:07.925783 26533 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:04:07.961328 27212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:07.937373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.131:36839 (local address 127.0.0.1:57364)
0811 12:04:07.951066 (+ 13693us) negotiation.cc:107] Waiting for socket to connect
0811 12:04:07.951104 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 12:04:07.951288 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:04:07.953494 (+ 2206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:04:07.953499 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:04:07.953525 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:04:07.953907 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:04:07.953919 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:04:07.956066 (+ 2147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:04:07.956072 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:04:07.958650 (+ 2578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:04:07.958662 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:04:07.960167 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:04:07.960174 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:04:07.960395 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:07.960465 (+ 70us) client_negotiation.cc:769] Sending connection context
0811 12:04:07.960622 (+ 157us) client_negotiation.cc:240] Negotiation successful
0811 12:04:07.960823 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13446,"mutex_wait_us":141,"thread_start_us":13296,"threads_started":1}
I20260811 12:04:07.961409 27214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:07.937674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57364 (local address 127.9.221.131:36839)
0811 12:04:07.952891 (+ 15217us) server_negotiation.cc:206] Beginning negotiation
0811 12:04:07.952901 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:04:07.952930 (+ 29us) server_negotiation.cc:407] Connection header received
0811 12:04:07.953021 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:04:07.953028 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:04:07.953269 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:04:07.954116 (+ 847us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:04:07.954389 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:07.955883 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:07.958819 (+ 2936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:07.959949 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:07.960100 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:07.960345 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0811 12:04:07.960844 (+ 499us) server_negotiation.cc:299] Negotiation successful
0811 12:04:07.961017 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15023,"thread_start_us":14836,"threads_started":1}
I20260811 12:04:07.961609 27215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:07.938058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35836 (local address 127.9.221.130:37087)
0811 12:04:07.950698 (+ 12640us) server_negotiation.cc:206] Beginning negotiation
0811 12:04:07.950711 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 12:04:07.951524 (+ 813us) server_negotiation.cc:407] Connection header received
0811 12:04:07.951628 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:04:07.951658 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:04:07.951913 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:04:07.952104 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:04:07.953027 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:07.954968 (+ 1941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:07.958867 (+ 3899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:07.960370 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:07.960544 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:07.960725 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 12:04:07.961062 (+ 337us) server_negotiation.cc:299] Negotiation successful
0811 12:04:07.961249 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12395,"thread_start_us":12179,"threads_started":1}
I20260811 12:04:07.962462 27213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:07.937529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.130:37087 (local address 127.0.0.1:35836)
0811 12:04:07.951282 (+ 13753us) negotiation.cc:107] Waiting for socket to connect
0811 12:04:07.951307 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 12:04:07.951472 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:04:07.952313 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:04:07.952320 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:04:07.952353 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:04:07.952802 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:04:07.952815 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:04:07.955861 (+ 3046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:04:07.955866 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:04:07.958677 (+ 2811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:04:07.958692 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:04:07.960606 (+ 1914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:04:07.960615 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:04:07.960813 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:07.960881 (+ 68us) client_negotiation.cc:769] Sending connection context
0811 12:04:07.961863 (+ 982us) client_negotiation.cc:240] Negotiation successful
0811 12:04:07.961964 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13585,"mutex_wait_us":177,"thread_start_us":14096,"threads_started":1}
I20260811 12:04:07.965720 27217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:07.952942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:37149 (local address 127.0.0.1:59510)
0811 12:04:07.956917 (+ 3975us) negotiation.cc:107] Waiting for socket to connect
0811 12:04:07.956941 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 12:04:07.957093 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:04:07.957679 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:04:07.957684 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:04:07.957708 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:04:07.958194 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:04:07.958209 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:04:07.960161 (+ 1952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:04:07.960166 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:04:07.962396 (+ 2230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:04:07.962412 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:04:07.964939 (+ 2527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:04:07.964946 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:04:07.965114 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:07.965144 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 12:04:07.965240 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 12:04:07.965356 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3786,"mutex_wait_us":2,"thread_start_us":3633,"threads_started":1}
I20260811 12:04:07.966056 27216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:07.952457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59510 (local address 127.9.221.129:37149)
0811 12:04:07.956279 (+ 3822us) server_negotiation.cc:206] Beginning negotiation
0811 12:04:07.956289 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:04:07.957075 (+ 786us) server_negotiation.cc:407] Connection header received
0811 12:04:07.957253 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:04:07.957263 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:04:07.957472 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:04:07.957659 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:04:07.958415 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:07.959968 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:07.962605 (+ 2637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:07.964722 (+ 2117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:07.964865 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:07.965076 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 12:04:07.965298 (+ 222us) server_negotiation.cc:299] Negotiation successful
0811 12:04:07.965466 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3649,"thread_start_us":3442,"threads_started":1}
I20260811 12:04:07.965987 26911 tablet_service.cc:1467] Processing CreateTablet for tablet 97a2c677977d46f38640766962cfc80f (DEFAULT_TABLE table=test-table [id=5f0f181f9ce14a06be71055ececa6c28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:04:07.967257 26911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97a2c677977d46f38640766962cfc80f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:04:07.965996 27092 tablet_service.cc:1467] Processing CreateTablet for tablet 97a2c677977d46f38640766962cfc80f (DEFAULT_TABLE table=test-table [id=5f0f181f9ce14a06be71055ececa6c28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:04:07.968144 27092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97a2c677977d46f38640766962cfc80f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:04:07.969368 26716 tablet_service.cc:1467] Processing CreateTablet for tablet 97a2c677977d46f38640766962cfc80f (DEFAULT_TABLE table=test-table [id=5f0f181f9ce14a06be71055ececa6c28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:04:07.970700 26716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97a2c677977d46f38640766962cfc80f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:04:07.978724 27219 tablet_bootstrap.cc:492] T 97a2c677977d46f38640766962cfc80f P a31eff20048d47099f7183ff9c0990c8: Bootstrap starting.
I20260811 12:04:07.981820 27219 tablet_bootstrap.cc:654] T 97a2c677977d46f38640766962cfc80f P a31eff20048d47099f7183ff9c0990c8: Neither blocks nor log segments found. Creating new log.
I20260811 12:04:07.983103 27219 log.cc:824] T 97a2c677977d46f38640766962cfc80f P a31eff20048d47099f7183ff9c0990c8: Log is configured to *not* fsync() on all Append() calls
I20260811 12:04:07.984311 27221 tablet_bootstrap.cc:492] T 97a2c677977d46f38640766962cfc80f P 0299f45f4e9e4affaff259752d1228f5: Bootstrap starting.
I20260811 12:04:07.986018 27225 tablet_bootstrap.cc:492] T 97a2c677977d46f38640766962cfc80f P a9a70fd2831c40269b8c296dd3a3b771: Bootstrap starting.
I20260811 12:04:07.987872 27219 tablet_bootstrap.cc:492] T 97a2c677977d46f38640766962cfc80f P a31eff20048d47099f7183ff9c0990c8: No bootstrap required, opened a new log
I20260811 12:04:07.988008 27221 tablet_bootstrap.cc:654] T 97a2c677977d46f38640766962cfc80f P 0299f45f4e9e4affaff259752d1228f5: Neither blocks nor log segments found. Creating new log.
I20260811 12:04:07.988178 27219 ts_tablet_manager.cc:1397] T 97a2c677977d46f38640766962cfc80f P a31eff20048d47099f7183ff9c0990c8: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260811 12:04:07.989174 27221 log.cc:824] T 97a2c677977d46f38640766962cfc80f P 0299f45f4e9e4affaff259752d1228f5: Log is configured to *not* fsync() on all Append() calls
I20260811 12:04:07.989444 27225 tablet_bootstrap.cc:654] T 97a2c677977d46f38640766962cfc80f P a9a70fd2831c40269b8c296dd3a3b771: Neither blocks nor log segments found. Creating new log.
I20260811 12:04:07.990447 27225 log.cc:824] T 97a2c677977d46f38640766962cfc80f P a9a70fd2831c40269b8c296dd3a3b771: Log is configured to *not* fsync() on all Append() calls
I20260811 12:04:07.992435 27221 tablet_bootstrap.cc:492] T 97a2c677977d46f38640766962cfc80f P 0299f45f4e9e4affaff259752d1228f5: No bootstrap required, opened a new log
I20260811 12:04:07.992688 27221 ts_tablet_manager.cc:1397] T 97a2c677977d46f38640766962cfc80f P 0299f45f4e9e4affaff259752d1228f5: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260811 12:04:07.993353 27225 tablet_bootstrap.cc:492] T 97a2c677977d46f38640766962cfc80f P a9a70fd2831c40269b8c296dd3a3b771: No bootstrap required, opened a new log
I20260811 12:04:07.993649 27225 ts_tablet_manager.cc:1397] T 97a2c677977d46f38640766962cfc80f P a9a70fd2831c40269b8c296dd3a3b771: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260811 12:04:07.998391 27219 raft_consensus.cc:348] T 97a2c677977d46f38640766962cfc80f P a31eff20048d47099f7183ff9c0990c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0299f45f4e9e4affaff259752d1228f5" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 36839 } } peers { permanent_uuid: "a31eff20048d47099f7183ff9c0990c8" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 37087 } } peers { permanent_uuid: "a9a70fd2831c40269b8c296dd3a3b771" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37149 } }
I20260811 12:04:07.998834 27219 raft_consensus.cc:374] T 97a2c677977d46f38640766962cfc80f P a31eff20048d47099f7183ff9c0990c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:04:07.998989 27219 raft_consensus.cc:729] T 97a2c677977d46f38640766962cfc80f P a31eff20048d47099f7183ff9c0990c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a31eff20048d47099f7183ff9c0990c8, State: Initialized, Role: FOLLOWER
I20260811 12:04:07.999619 27219 consensus_queue.cc:260] T 97a2c677977d46f38640766962cfc80f P a31eff20048d47099f7183ff9c0990c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0299f45f4e9e4affaff259752d1228f5" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 36839 } } peers { permanent_uuid: "a31eff20048d47099f7183ff9c0990c8" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 37087 } } peers { permanent_uuid: "a9a70fd2831c40269b8c296dd3a3b771" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37149 } }
I20260811 12:04:07.999629 27221 raft_consensus.cc:348] T 97a2c677977d46f38640766962cfc80f P 0299f45f4e9e4affaff259752d1228f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0299f45f4e9e4affaff259752d1228f5" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 36839 } } peers { permanent_uuid: "a31eff20048d47099f7183ff9c0990c8" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 37087 } } peers { permanent_uuid: "a9a70fd2831c40269b8c296dd3a3b771" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37149 } }
I20260811 12:04:08.000000 27221 raft_consensus.cc:374] T 97a2c677977d46f38640766962cfc80f P 0299f45f4e9e4affaff259752d1228f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:04:08.000134 27221 raft_consensus.cc:729] T 97a2c677977d46f38640766962cfc80f P 0299f45f4e9e4affaff259752d1228f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0299f45f4e9e4affaff259752d1228f5, State: Initialized, Role: FOLLOWER
I20260811 12:04:08.000872 26998 heartbeater.cc:503] Master 127.9.221.190:42107 was elected leader, sending a full tablet report...
I20260811 12:04:08.001636 27219 ts_tablet_manager.cc:1428] T 97a2c677977d46f38640766962cfc80f P a31eff20048d47099f7183ff9c0990c8: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260811 12:04:08.002090 27221 consensus_queue.cc:260] T 97a2c677977d46f38640766962cfc80f P 0299f45f4e9e4affaff259752d1228f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0299f45f4e9e4affaff259752d1228f5" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 36839 } } peers { permanent_uuid: "a31eff20048d47099f7183ff9c0990c8" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 37087 } } peers { permanent_uuid: "a9a70fd2831c40269b8c296dd3a3b771" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37149 } }
I20260811 12:04:08.004621 27225 raft_consensus.cc:348] T 97a2c677977d46f38640766962cfc80f P a9a70fd2831c40269b8c296dd3a3b771 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0299f45f4e9e4affaff259752d1228f5" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 36839 } } peers { permanent_uuid: "a31eff20048d47099f7183ff9c0990c8" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 37087 } } peers { permanent_uuid: "a9a70fd2831c40269b8c296dd3a3b771" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37149 } }
I20260811 12:04:08.005023 27225 raft_consensus.cc:374] T 97a2c677977d46f38640766962cfc80f P a9a70fd2831c40269b8c296dd3a3b771 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:04:08.005120 27225 raft_consensus.cc:729] T 97a2c677977d46f38640766962cfc80f P a9a70fd2831c40269b8c296dd3a3b771 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a70fd2831c40269b8c296dd3a3b771, State: Initialized, Role: FOLLOWER
I20260811 12:04:08.005703 27192 heartbeater.cc:503] Master 127.9.221.190:42107 was elected leader, sending a full tablet report...
I20260811 12:04:08.005800 27225 consensus_queue.cc:260] T 97a2c677977d46f38640766962cfc80f P a9a70fd2831c40269b8c296dd3a3b771 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0299f45f4e9e4affaff259752d1228f5" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 36839 } } peers { permanent_uuid: "a31eff20048d47099f7183ff9c0990c8" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 37087 } } peers { permanent_uuid: "a9a70fd2831c40269b8c296dd3a3b771" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37149 } }
I20260811 12:04:08.005716 27221 ts_tablet_manager.cc:1428] T 97a2c677977d46f38640766962cfc80f P 0299f45f4e9e4affaff259752d1228f5: Time spent starting tablet: real 0.013s user 0.009s sys 0.002s
I20260811 12:04:08.008471 27225 ts_tablet_manager.cc:1428] T 97a2c677977d46f38640766962cfc80f P a9a70fd2831c40269b8c296dd3a3b771: Time spent starting tablet: real 0.015s user 0.010s sys 0.005s
I20260811 12:04:08.051963 27228 raft_consensus.cc:482] T 97a2c677977d46f38640766962cfc80f P a31eff20048d47099f7183ff9c0990c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:04:08.052278 27228 raft_consensus.cc:504] T 97a2c677977d46f38640766962cfc80f P a31eff20048d47099f7183ff9c0990c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0299f45f4e9e4affaff259752d1228f5" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 36839 } } peers { permanent_uuid: "a31eff20048d47099f7183ff9c0990c8" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 37087 } } peers { permanent_uuid: "a9a70fd2831c40269b8c296dd3a3b771" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37149 } }
W20260811 12:04:08.053390 26818 tablet.cc:2369] T 97a2c677977d46f38640766962cfc80f P a9a70fd2831c40269b8c296dd3a3b771: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:04:08.054050 27228 leader_election.cc:288] T 97a2c677977d46f38640766962cfc80f P a31eff20048d47099f7183ff9c0990c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0299f45f4e9e4affaff259752d1228f5 (127.9.221.131:36839), a9a70fd2831c40269b8c296dd3a3b771 (127.9.221.129:37149)
I20260811 12:04:08.060328 27214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:08.054473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:36091 (local address 127.9.221.131:36839)
0811 12:04:08.054691 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 12:04:08.054697 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:04:08.055100 (+ 403us) server_negotiation.cc:407] Connection header received
0811 12:04:08.055327 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:04:08.055332 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:04:08.055395 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:04:08.055487 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:04:08.056029 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:08.056793 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:08.058632 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:08.059391 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:08.059589 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:08.059682 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 12:04:08.059953 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 12:04:08.060078 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 12:04:08.060397 27232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:08.054188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.131:36839 (local address 127.9.221.130:36091)
0811 12:04:08.054992 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0811 12:04:08.055008 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:04:08.055174 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:04:08.055509 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:04:08.055513 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:04:08.055541 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:04:08.055886 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:04:08.055895 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:04:08.056928 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:04:08.056935 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:04:08.058478 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:04:08.058491 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:04:08.059607 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:04:08.059613 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:04:08.059787 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:08.059810 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 12:04:08.059910 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 12:04:08.060041 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":640,"thread_start_us":798,"threads_started":1}
I20260811 12:04:08.060722 27216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:08.054623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:54421 (local address 127.9.221.129:37149)
0811 12:04:08.054864 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 12:04:08.054870 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:04:08.055543 (+ 673us) server_negotiation.cc:407] Connection header received
0811 12:04:08.055780 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:04:08.055784 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:04:08.055839 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:04:08.055933 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:04:08.056472 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:08.057177 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:08.058986 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:08.059808 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:08.060000 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:08.060073 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 12:04:08.060355 (+ 282us) server_negotiation.cc:299] Negotiation successful
0811 12:04:08.060480 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 12:04:08.060830 27233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:08.054682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:37149 (local address 127.9.221.130:54421)
0811 12:04:08.055423 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0811 12:04:08.055437 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 12:04:08.055604 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:04:08.055948 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:04:08.055952 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:04:08.055974 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:04:08.056321 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:04:08.056329 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:04:08.057332 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:04:08.057337 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:04:08.058811 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:04:08.058824 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:04:08.060023 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:04:08.060029 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:04:08.060201 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:08.060224 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 12:04:08.060329 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 12:04:08.060467 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":639,"thread_start_us":433,"threads_started":1}
I20260811 12:04:08.061703 27128 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97a2c677977d46f38640766962cfc80f" candidate_uuid: "a31eff20048d47099f7183ff9c0990c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0299f45f4e9e4affaff259752d1228f5" is_pre_election: true
I20260811 12:04:08.061812 26749 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97a2c677977d46f38640766962cfc80f" candidate_uuid: "a31eff20048d47099f7183ff9c0990c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a70fd2831c40269b8c296dd3a3b771" is_pre_election: true
I20260811 12:04:08.062340 27128 raft_consensus.cc:2454] T 97a2c677977d46f38640766962cfc80f P 0299f45f4e9e4affaff259752d1228f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a31eff20048d47099f7183ff9c0990c8 in term 0.
I20260811 12:04:08.062340 26749 raft_consensus.cc:2454] T 97a2c677977d46f38640766962cfc80f P a9a70fd2831c40269b8c296dd3a3b771 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a31eff20048d47099f7183ff9c0990c8 in term 0.
I20260811 12:04:08.063048 26853 leader_election.cc:302] T 97a2c677977d46f38640766962cfc80f P a31eff20048d47099f7183ff9c0990c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0299f45f4e9e4affaff259752d1228f5, a31eff20048d47099f7183ff9c0990c8; no voters:
I20260811 12:04:08.063474 27228 raft_consensus.cc:2781] T 97a2c677977d46f38640766962cfc80f P a31eff20048d47099f7183ff9c0990c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:04:08.063621 27228 raft_consensus.cc:482] T 97a2c677977d46f38640766962cfc80f P a31eff20048d47099f7183ff9c0990c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:04:08.063767 27228 raft_consensus.cc:3037] T 97a2c677977d46f38640766962cfc80f P a31eff20048d47099f7183ff9c0990c8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:04:08.066114 27228 raft_consensus.cc:504] T 97a2c677977d46f38640766962cfc80f P a31eff20048d47099f7183ff9c0990c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0299f45f4e9e4affaff259752d1228f5" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 36839 } } peers { permanent_uuid: "a31eff20048d47099f7183ff9c0990c8" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 37087 } } peers { permanent_uuid: "a9a70fd2831c40269b8c296dd3a3b771" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37149 } }
I20260811 12:04:08.066779 27228 leader_election.cc:288] T 97a2c677977d46f38640766962cfc80f P a31eff20048d47099f7183ff9c0990c8 [CANDIDATE]: Term 1 election: Requested vote from peers 0299f45f4e9e4affaff259752d1228f5 (127.9.221.131:36839), a9a70fd2831c40269b8c296dd3a3b771 (127.9.221.129:37149)
I20260811 12:04:08.067114 27128 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97a2c677977d46f38640766962cfc80f" candidate_uuid: "a31eff20048d47099f7183ff9c0990c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0299f45f4e9e4affaff259752d1228f5"
I20260811 12:04:08.067260 26749 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97a2c677977d46f38640766962cfc80f" candidate_uuid: "a31eff20048d47099f7183ff9c0990c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9a70fd2831c40269b8c296dd3a3b771"
I20260811 12:04:08.067319 27128 raft_consensus.cc:3037] T 97a2c677977d46f38640766962cfc80f P 0299f45f4e9e4affaff259752d1228f5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:04:08.067480 26749 raft_consensus.cc:3037] T 97a2c677977d46f38640766962cfc80f P a9a70fd2831c40269b8c296dd3a3b771 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:04:08.070638 26749 raft_consensus.cc:2454] T 97a2c677977d46f38640766962cfc80f P a9a70fd2831c40269b8c296dd3a3b771 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a31eff20048d47099f7183ff9c0990c8 in term 1.
I20260811 12:04:08.070638 27128 raft_consensus.cc:2454] T 97a2c677977d46f38640766962cfc80f P 0299f45f4e9e4affaff259752d1228f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a31eff20048d47099f7183ff9c0990c8 in term 1.
I20260811 12:04:08.071041 26853 leader_election.cc:302] T 97a2c677977d46f38640766962cfc80f P a31eff20048d47099f7183ff9c0990c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0299f45f4e9e4affaff259752d1228f5, a31eff20048d47099f7183ff9c0990c8; no voters:
I20260811 12:04:08.071344 27228 raft_consensus.cc:2781] T 97a2c677977d46f38640766962cfc80f P a31eff20048d47099f7183ff9c0990c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:04:08.071852 27228 raft_consensus.cc:686] T 97a2c677977d46f38640766962cfc80f P a31eff20048d47099f7183ff9c0990c8 [term 1 LEADER]: Becoming Leader. State: Replica: a31eff20048d47099f7183ff9c0990c8, State: Running, Role: LEADER
I20260811 12:04:08.072304 27228 consensus_queue.cc:237] T 97a2c677977d46f38640766962cfc80f P a31eff20048d47099f7183ff9c0990c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0299f45f4e9e4affaff259752d1228f5" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 36839 } } peers { permanent_uuid: "a31eff20048d47099f7183ff9c0990c8" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 37087 } } peers { permanent_uuid: "a9a70fd2831c40269b8c296dd3a3b771" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37149 } }
I20260811 12:04:08.076928 26531 catalog_manager.cc:5495] T 97a2c677977d46f38640766962cfc80f P a31eff20048d47099f7183ff9c0990c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a31eff20048d47099f7183ff9c0990c8 (127.9.221.130). New cstate: current_term: 1 leader_uuid: "a31eff20048d47099f7183ff9c0990c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0299f45f4e9e4affaff259752d1228f5" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 36839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a31eff20048d47099f7183ff9c0990c8" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 37087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9a70fd2831c40269b8c296dd3a3b771" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37149 } health_report { overall_health: UNKNOWN } } }
W20260811 12:04:08.081598 26999 tablet.cc:2369] T 97a2c677977d46f38640766962cfc80f P a31eff20048d47099f7183ff9c0990c8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:04:08.114095 27193 tablet.cc:2369] T 97a2c677977d46f38640766962cfc80f P 0299f45f4e9e4affaff259752d1228f5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:04:08.117266 27215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:08.111869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35840 (local address 127.9.221.130:37087)
0811 12:04:08.112070 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 12:04:08.112075 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:04:08.112099 (+ 24us) server_negotiation.cc:407] Connection header received
0811 12:04:08.112228 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:04:08.112231 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:04:08.112305 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:04:08.112423 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:04:08.112734 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:08.113418 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:08.115047 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:08.115534 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:08.115717 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:08.116196 (+ 479us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:04:08.116675 (+ 479us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:04:08.116836 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 12:04:08.116914 (+ 78us) server_negotiation.cc:299] Negotiation successful
0811 12:04:08.116996 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 12:04:08.125367 27128 raft_consensus.cc:1261] T 97a2c677977d46f38640766962cfc80f P 0299f45f4e9e4affaff259752d1228f5 [term 1 FOLLOWER]: Refusing update from remote peer a31eff20048d47099f7183ff9c0990c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:04:08.125428 26749 raft_consensus.cc:1261] T 97a2c677977d46f38640766962cfc80f P a9a70fd2831c40269b8c296dd3a3b771 [term 1 FOLLOWER]: Refusing update from remote peer a31eff20048d47099f7183ff9c0990c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:04:08.126358 27228 consensus_queue.cc:1035] T 97a2c677977d46f38640766962cfc80f P a31eff20048d47099f7183ff9c0990c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0299f45f4e9e4affaff259752d1228f5" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 36839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:04:08.126688 27234 consensus_queue.cc:1035] T 97a2c677977d46f38640766962cfc80f P a31eff20048d47099f7183ff9c0990c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9a70fd2831c40269b8c296dd3a3b771" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 37149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:04:08.138479 27240 mvcc.cc:205] Tried to move back new op lower bound from 7317298577910833152 to 7317298577708539904. Current Snapshot: MvccSnapshot[applied={T|T < 7317298577910833152}]
I20260811 12:04:08.153541 26530 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42328:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:04:08.153785 26530 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42328:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:04:08.157601 26530 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d11e9824bd6f4eb7b9f04021343ea734: Sending DeleteTablet for 3 replicas of tablet 97a2c677977d46f38640766962cfc80f
I20260811 12:04:08.158972 26911 tablet_service.cc:1514] Processing DeleteTablet for tablet 97a2c677977d46f38640766962cfc80f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:04:08 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:35836
I20260811 12:04:08.159003 26716 tablet_service.cc:1514] Processing DeleteTablet for tablet 97a2c677977d46f38640766962cfc80f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:04:08 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:59510
I20260811 12:04:08.160398 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 26628
I20260811 12:04:08.160789 27092 tablet_service.cc:1514] Processing DeleteTablet for tablet 97a2c677977d46f38640766962cfc80f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:04:08 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:57364
I20260811 12:04:08.175987 27255 tablet_replica.cc:330] T 97a2c677977d46f38640766962cfc80f P 0299f45f4e9e4affaff259752d1228f5: stopping tablet replica
I20260811 12:04:08.176532 27255 raft_consensus.cc:2229] T 97a2c677977d46f38640766962cfc80f P 0299f45f4e9e4affaff259752d1228f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:04:08.177096 27255 raft_consensus.cc:2258] T 97a2c677977d46f38640766962cfc80f P 0299f45f4e9e4affaff259752d1228f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:04:08.178413 27253 tablet_replica.cc:330] T 97a2c677977d46f38640766962cfc80f P a31eff20048d47099f7183ff9c0990c8: stopping tablet replica
I20260811 12:04:08.178963 27254 tablet_replica.cc:330] T 97a2c677977d46f38640766962cfc80f P a9a70fd2831c40269b8c296dd3a3b771: stopping tablet replica
I20260811 12:04:08.179518 27254 raft_consensus.cc:2229] T 97a2c677977d46f38640766962cfc80f P a9a70fd2831c40269b8c296dd3a3b771 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:04:08.179986 27255 ts_tablet_manager.cc:1905] T 97a2c677977d46f38640766962cfc80f P 0299f45f4e9e4affaff259752d1228f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:04:08.180760 27253 raft_consensus.cc:2229] T 97a2c677977d46f38640766962cfc80f P a31eff20048d47099f7183ff9c0990c8 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:04:08.181617 27253 raft_consensus.cc:2258] T 97a2c677977d46f38640766962cfc80f P a31eff20048d47099f7183ff9c0990c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:04:08.182192 27254 raft_consensus.cc:2258] T 97a2c677977d46f38640766962cfc80f P a9a70fd2831c40269b8c296dd3a3b771 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:04:08.184441 27254 ts_tablet_manager.cc:1905] T 97a2c677977d46f38640766962cfc80f P a9a70fd2831c40269b8c296dd3a3b771: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:04:08.186209 27253 ts_tablet_manager.cc:1905] T 97a2c677977d46f38640766962cfc80f P a31eff20048d47099f7183ff9c0990c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:04:08.186988 27255 ts_tablet_manager.cc:1918] T 97a2c677977d46f38640766962cfc80f P 0299f45f4e9e4affaff259752d1228f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:04:08.187178 27255 log.cc:1196] T 97a2c677977d46f38640766962cfc80f P 0299f45f4e9e4affaff259752d1228f5: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-2/wal/wals/97a2c677977d46f38640766962cfc80f
I20260811 12:04:08.187777 27255 ts_tablet_manager.cc:1939] T 97a2c677977d46f38640766962cfc80f P 0299f45f4e9e4affaff259752d1228f5: Deleting consensus metadata
I20260811 12:04:08.189594 26509 catalog_manager.cc:4841] TS 0299f45f4e9e4affaff259752d1228f5 (127.9.221.131:36839): tablet 97a2c677977d46f38640766962cfc80f (table test-table [id=5f0f181f9ce14a06be71055ececa6c28]) successfully deleted
Aug 11 12:04:08 dist-test-slave-mjjr krb5kdc[26461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449845, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.129@KRBTEST.COM
I20260811 12:04:08.195561 27254 ts_tablet_manager.cc:1918] T 97a2c677977d46f38640766962cfc80f P a9a70fd2831c40269b8c296dd3a3b771: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:04:08.195848 27254 log.cc:1196] T 97a2c677977d46f38640766962cfc80f P a9a70fd2831c40269b8c296dd3a3b771: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-0/wal/wals/97a2c677977d46f38640766962cfc80f
I20260811 12:04:08.196419 27254 ts_tablet_manager.cc:1939] T 97a2c677977d46f38640766962cfc80f P a9a70fd2831c40269b8c296dd3a3b771: Deleting consensus metadata
I20260811 12:04:08.198387 26512 catalog_manager.cc:4841] TS a9a70fd2831c40269b8c296dd3a3b771 (127.9.221.129:37149): tablet 97a2c677977d46f38640766962cfc80f (table test-table [id=5f0f181f9ce14a06be71055ececa6c28]) successfully deleted
I20260811 12:04:08.199733 27216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:08.162181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59520 (local address 127.9.221.129:37149)
0811 12:04:08.162406 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 12:04:08.162437 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0811 12:04:08.172703 (+ 10266us) server_negotiation.cc:407] Connection header received
0811 12:04:08.173261 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:04:08.173270 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:04:08.173383 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:04:08.173483 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:04:08.175276 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:08.176556 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:08.179074 (+ 2518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:08.179853 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:08.179957 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:08.191863 (+ 11906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:04:08.191876 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:04:08.191893 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:04:08.191947 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:04:08.195842 (+ 3895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:04:08.196699 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:04:08.196708 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:04:08.196712 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:04:08.196887 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:04:08.197343 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:04:08.197348 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:04:08.197352 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:04:08.197779 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:04:08.198082 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0811 12:04:08.199158 (+ 1076us) server_negotiation.cc:299] Negotiation successful
0811 12:04:08.199361 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 12:04:08.200378 26811 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:04:08.205678 27253 ts_tablet_manager.cc:1918] T 97a2c677977d46f38640766962cfc80f P a31eff20048d47099f7183ff9c0990c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:04:08.205885 27253 log.cc:1196] T 97a2c677977d46f38640766962cfc80f P a31eff20048d47099f7183ff9c0990c8: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449743084027-10102-0/minicluster-data/ts-1/wal/wals/97a2c677977d46f38640766962cfc80f
I20260811 12:04:08.206513 27253 ts_tablet_manager.cc:1939] T 97a2c677977d46f38640766962cfc80f P a31eff20048d47099f7183ff9c0990c8: Deleting consensus metadata
I20260811 12:04:08.208412 26512 catalog_manager.cc:4841] TS a31eff20048d47099f7183ff9c0990c8 (127.9.221.130:37087): tablet 97a2c677977d46f38640766962cfc80f (table test-table [id=5f0f181f9ce14a06be71055ececa6c28]) successfully deleted
I20260811 12:04:08.291412 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 26628
I20260811 12:04:08.317453 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 26826
Aug 11 12:04:08 dist-test-slave-mjjr krb5kdc[26461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449845, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.130@KRBTEST.COM
I20260811 12:04:08.337030 27215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:08.318322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35850 (local address 127.9.221.130:37087)
0811 12:04:08.318526 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 12:04:08.318531 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:04:08.318693 (+ 162us) server_negotiation.cc:407] Connection header received
0811 12:04:08.318883 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:04:08.318887 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:04:08.319003 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:04:08.319106 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:04:08.320779 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:08.321625 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:08.323475 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:08.323972 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:08.324064 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:08.329494 (+ 5430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:04:08.329509 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:04:08.329525 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:04:08.329578 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:04:08.333675 (+ 4097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:04:08.334423 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:04:08.334432 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:04:08.334435 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:04:08.334655 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:04:08.335049 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:04:08.335054 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:04:08.335057 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:04:08.335533 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:04:08.336103 (+ 570us) server_negotiation.cc:1035] Waiting for connection context
0811 12:04:08.336383 (+ 280us) server_negotiation.cc:299] Negotiation successful
0811 12:04:08.336621 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 12:04:08.337807 26992 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:04:08.449474 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 26826
I20260811 12:04:08.474933 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 27010
Aug 11 12:04:08 dist-test-slave-mjjr krb5kdc[26461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449845, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.131@KRBTEST.COM
I20260811 12:04:08.489677 27214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:08.475806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57380 (local address 127.9.221.131:36839)
0811 12:04:08.475969 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 12:04:08.475974 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:04:08.475992 (+ 18us) server_negotiation.cc:407] Connection header received
0811 12:04:08.476094 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:04:08.476098 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:04:08.476160 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:04:08.476237 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:04:08.477555 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:08.478262 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:08.479544 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:08.480012 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:08.480106 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:08.484391 (+ 4285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:04:08.484404 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:04:08.484419 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:04:08.484458 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:04:08.487375 (+ 2917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:04:08.488022 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:04:08.488029 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:04:08.488032 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:04:08.488169 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:04:08.488487 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:04:08.488490 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:04:08.488492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:04:08.488839 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:04:08.489071 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 12:04:08.489262 (+ 191us) server_negotiation.cc:299] Negotiation successful
0811 12:04:08.489425 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 12:04:08.490327 27185 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:04:08.579696 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 27010
I20260811 12:04:08.603233 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 26485
I20260811 12:04:08.604405 26583 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:04:08.715117 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 26485
2026-08-11T12:04:08Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4620 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:04:08 dist-test-slave-mjjr krb5kdc[27266](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:04:08 dist-test-slave-mjjr krb5kdc[27266](info): set up 2 sockets
Aug 11 12:04:08 dist-test-slave-mjjr krb5kdc[27266](info): commencing operation
krb5kdc: starting...
W20260811 12:04:10.037813 10102 mini_kdc.cc:121] Time spent starting KDC: real 1.268s user 0.003s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Aug 11 12:04:10 dist-test-slave-mjjr krb5kdc[27266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449850, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:04:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:04:10Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.9.221.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.9.221.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.9.221.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/krb5kdc/rangeradmin_127.9.221.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.9.221.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/krb5kdc/rangeradmin_127.9.221.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.9.221.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.9.221.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.9.221.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/krb5kdc/rangerlookup_127.9.221.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.9.221.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/krb5kdc/rangerlookup_127.9.221.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.221.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.9.221.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.221.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/krb5kdc/HTTP_127.9.221.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.221.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/krb5kdc/HTTP_127.9.221.148@KRBTEST.COM.keytab.
I20260811 12:04:10.388113 10102 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskGvgjmQ/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/postgres -l logfile start
2026-08-11 12:04:12.612 UTC [27308] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 12:04:12.613 UTC [27308] LOG: listening on IPv4 address "127.9.221.148", port 43157
2026-08-11 12:04:12.616 UTC [27308] LOG: listening on Unix socket "/tmp/.s.PGSQL.43157"
2026-08-11 12:04:12.640 UTC [27311] LOG: database system was shut down at 2026-08-11 12:04:11 UTC
2026-08-11 12:04:12.643 UTC [27308] LOG: database system is ready to accept connections
I20260811 12:04:13.898412 10102 mini_postgres.cc:96] Postgres bound to 43157
2026-08-11 12:04:13.921 UTC [27324] FATAL: database "slave" does not exist
127.9.221.148:43157 - accepting connections
I20260811 12:04:13.922801 10102 mini_ranger.cc:161] Starting Ranger...
I20260811 12:04:13.952006 10102 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 12:04:14.367309 10102 mini_ranger.cc:87] Created ranger Postgres database
I20260811 12:04:14.367501 10102 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ranger-admin
2026-08-11 12:04:14,707 [I] DB FLAVOR :POSTGRES
2026-08-11 12:04:14,708 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 12:04:14,708 [I] Checking connection..
2026-08-11 12:04:14,708 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.221.148:43157/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 12:04:14,971 [I] Checking connection passed.
2026-08-11 12:04:14,971 [I] --------- Verifying version history table ---------
2026-08-11 12:04:14,972 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.221.148:43157/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 12:04:15,229 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 12:04:15,229 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.221.148:43157/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 12:04:15,468 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 12:04:15,468 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 12:04:15,469 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.221.148:43157/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 12:04:15.706 UTC [27409] WARNING: there is no transaction in progress
2026-08-11 12:04:15,715 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 12:04:15,716 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.221.148:43157/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 12:04:15,978 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 12:04:15,978 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 12:04:15,978 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.221.148:43157/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 12:04:16,240 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.221.148:43157/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 12:04:16,483 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.221.148:43157/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-mjjr.c.gcp-upstream.internal','N') ;"
2026-08-11 12:04:16,710 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.221.148:43157/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 12:04:16,969 [I] Table x_portal_user does not exist in database ranger
2026-08-11 12:04:16,969 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.221.148:43157/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 12:04:17,250 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 12:04:17,251 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.221.148:43157/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 12:04:17,518 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.221.148:43157/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 12:04:17,811 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 12:04:17,812 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.221.148:43157/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 12:04:18.709 UTC [27570] WARNING: there is no transaction in progress
2026-08-11 12:04:18.724 UTC [27570] WARNING: there is no transaction in progress
2026-08-11 12:04:18.740 UTC [27570] WARNING: there is no transaction in progress
2026-08-11 12:04:18.756 UTC [27570] WARNING: there is no transaction in progress
2026-08-11 12:04:18.773 UTC [27570] WARNING: there is no transaction in progress
2026-08-11 12:04:18.788 UTC [27570] WARNING: there is no transaction in progress
2026-08-11 12:04:18.840 UTC [27570] WARNING: there is no transaction in progress
2026-08-11 12:04:18.846 UTC [27570] WARNING: there is no transaction in progress
2026-08-11 12:04:18.859 UTC [27570] WARNING: there is no transaction in progress
2026-08-11 12:04:18.868 UTC [27570] WARNING: there is no transaction in progress
2026-08-11 12:04:18.878 UTC [27570] WARNING: there is no transaction in progress
2026-08-11 12:04:18.890 UTC [27570] WARNING: there is no transaction in progress
2026-08-11 12:04:18.900 UTC [27570] WARNING: there is no transaction in progress
2026-08-11 12:04:18.906 UTC [27570] WARNING: there is no transaction in progress
2026-08-11 12:04:18.912 UTC [27570] WARNING: there is no transaction in progress
2026-08-11 12:04:18.918 UTC [27570] WARNING: there is no transaction in progress
2026-08-11 12:04:19,498 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 12:04:19,498 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.221.148:43157/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 12:04:19,772 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 12:04:19,772 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.221.148:43157/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 12:04:20,044 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 12:04:20,044 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.221.148:43157/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 12:04:20,311 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 12:04:20,311 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.221.148:43157/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 12:04:20,555 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.221.148:43157/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 12:04:20,806 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.221.148:43157/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-mjjr.c.gcp-upstream.internal';"
2026-08-11 12:04:21,060 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 12:04:21,060 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.221.148:43157/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 12:04:21,323 [I] DB_PATCHES have already been applied
I20260811 12:04:21.328898 10102 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ranger-admin:/tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskGvgjmQ/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 12:04:21.329012 10102 mini_ranger.cc:193] Using host: 127.9.221.148
I20260811 12:04:21.349308 10102 mini_ranger.cc:227] Ranger admin URL: http://127.9.221.148:53055
Aug 11, 2026 12:04:21 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 12:04:21 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 12:04:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 12:04:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 12:04:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 12:04:22 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 12:04:22 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 12:04:22.731966 10102 mini_ranger.cc:160] Time spent starting Ranger: real 8.809s user 0.010s sys 0.047s
Aug 11, 2026 12:04:43 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 12:04:43 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 11, 2026 12:04:43 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 12:04:43 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 12:04:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 12:04:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 12:04:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 12:04:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 12:04:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 12:04:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 12:04:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 12:04:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 12:04:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 12:04:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 12:04:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 12:04:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 12:04:43 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 12:04:44 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 12:04:44.802772 10102 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.9.221.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.9.221.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.9.221.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/krb5kdc/rangerkms_127.9.221.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.9.221.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/krb5kdc/rangerkms_127.9.221.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.221.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.9.221.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.221.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/krb5kdc/HTTP_127.9.221.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.221.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/krb5kdc/HTTP_127.9.221.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 12:04:45 dist-test-slave-mjjr krb5kdc[27266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449885, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 12:04:45.015612 10102 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 12:04:45.042676 10102 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 12:04:45.783471 10102 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 12:04:45.783622 10102 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ranger-kms
I20260811 12:04:45.783808 10102 mini_ranger_kms.cc:227] Using postgres at 127.9.221.148:43157
2026-08-11 12:04:46,262 [I] DB FLAVOR :POSTGRES
2026-08-11 12:04:46,262 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 12:04:46,262 [I] Checking connection
2026-08-11 12:04:46,262 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.221.148:43157/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 12:04:46,507 [I] connection success
2026-08-11 12:04:46,507 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 12:04:46,507 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 12:04:46,508 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.221.148:43157/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 12:04:46,743 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 12:04:46,743 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 12:04:46,743 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 12:04:46,744 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.221.148:43157/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 12:04:46,997 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 12:04:47.506448 10102 mini_ranger_kms.cc:310] Created kms service
I20260811 12:04:47.712935 10102 mini_ranger_kms.cc:326] Created kudu user
I20260811 12:04:47.761087 10102 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 12:04:47 dist-test-slave-mjjr krb5kdc[27266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449885, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.9.221.148@KRBTEST.COM
I20260811 12:04:48.187944 10102 mini_ranger_kms.cc:384] Added ranger policy
I20260811 12:04:48.189388 10102 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ranger-kms:/tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 12:04:48.189474 10102 mini_ranger_kms.cc:240] Using host: 127.9.221.148
I20260811 12:04:48.219573 10102 mini_ranger_kms.cc:276] Ranger KMS PID: 27831
I20260811 12:04:48.219887 10102 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.9.221.148:44357
Aug 11, 2026 12:04:49 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 12:04:49 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 12:04:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 12:04:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 12:04:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 12:04:49 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 12:04:50 dist-test-slave-mjjr krb5kdc[27266](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786449890, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.221.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:04:51.043660 10102 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.028s user 0.116s sys 0.401s
I20260811 12:04:51.044059 10102 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 12:04:51.044291 10102 mini_ranger_kms.cc:401] 127.9.221.148:44357/kms/v1/keys
Aug 11 12:04:51 dist-test-slave-mjjr krb5kdc[27266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449890, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.221.148@KRBTEST.COM for HTTP/127.9.221.148@KRBTEST.COM
Aug 11 12:04:51 dist-test-slave-mjjr krb5kdc[27266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449890, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.221.148@KRBTEST.COM for HTTP/127.9.221.148@KRBTEST.COM
Aug 11 12:04:54 dist-test-slave-mjjr krb5kdc[27266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449894, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.9.221.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
I20260811 12:04:54.177817 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:46163
--webserver_interface=127.9.221.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:39815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:46163
--ranger_config_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.221.148:44357/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:04:54.694324 27907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:04:54.694729 27907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:04:54.694934 27907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:04:54.703887 27907 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:04:54.704062 27907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:04:54.704169 27907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:04:54.704331 27907 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:04:54.704479 27907 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:04:54.716862 27907 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:39815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/master-0/wal
--ranger_kms_url=127.9.221.148:44357/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:46163
--ranger_config_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:46163
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.221.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:04:54.718237 27907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:04:54.719887 27907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:04:54.735816 27914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:04:54.735765 27913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:04:54.736629 27918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:04:54.737391 27907 server_base.cc:1034] running on GCE node
I20260811 12:04:54.738255 27907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:04:54.739825 27907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:04:54.741086 27907 hybrid_clock.cc:648] HybridClock initialized: now 1786449894741000 us; error 51 us; skew 500 ppm
I20260811 12:04:54.741592 27907 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:04:54 dist-test-slave-mjjr krb5kdc[27266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449894, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:04:54.750469 27907 init.cc:377] Logged in from keytab as kudu/127.9.221.190@KRBTEST.COM (short username kudu)
I20260811 12:04:54.752748 27907 webserver.cc:460] Webserver started at http://127.9.221.190:38435/ using document root <none> and password file <none>
I20260811 12:04:54.753470 27907 fs_manager.cc:359] Metadata directory not provided
I20260811 12:04:54.753698 27907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:04:54.754067 27907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 12:04:54 dist-test-slave-mjjr krb5kdc[27266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449894, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for HTTP/127.9.221.148@KRBTEST.COM
I20260811 12:04:54.825098 27907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "0cb38209c8b04ac1a3a68103b6178202"
format_stamp: "Formatted at 2026-08-11 12:04:54 on dist-test-slave-mjjr"
server_key: "0ae5463a90aa4682c035681eb89d85d3"
server_key_iv: "f6abe304ded4e99b426fea9815f18c4c"
server_key_version: "kuduclusterkey@0"
I20260811 12:04:54.826810 27907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "0cb38209c8b04ac1a3a68103b6178202"
format_stamp: "Formatted at 2026-08-11 12:04:54 on dist-test-slave-mjjr"
server_key: "0ae5463a90aa4682c035681eb89d85d3"
server_key_iv: "f6abe304ded4e99b426fea9815f18c4c"
server_key_version: "kuduclusterkey@0"
I20260811 12:04:54.835610 27907 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.009s sys 0.001s
I20260811 12:04:54.858721 27929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:04:54.860672 27907 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 12:04:54.860940 27907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "0cb38209c8b04ac1a3a68103b6178202"
format_stamp: "Formatted at 2026-08-11 12:04:54 on dist-test-slave-mjjr"
server_key: "0ae5463a90aa4682c035681eb89d85d3"
server_key_iv: "f6abe304ded4e99b426fea9815f18c4c"
server_key_version: "kuduclusterkey@0"
I20260811 12:04:54.861332 27907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:04:54.908070 27907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:04:54.912823 27907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:04:54.913270 27907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:04:54.956599 27907 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.190:46163
I20260811 12:04:54.956601 27988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.190:46163 every 8 connection(s)
I20260811 12:04:54.958485 27907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:04:54.963490 27989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:04:54.966725 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 27907
I20260811 12:04:54.966953 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:04:54.967881 10102 external_mini_cluster.cc:1428] Setting key 20cf6c10ba806ca8ea1f423492b7aff9
I20260811 12:04:54.978540 27989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cb38209c8b04ac1a3a68103b6178202: Bootstrap starting.
I20260811 12:04:54.982547 27989 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0cb38209c8b04ac1a3a68103b6178202: Neither blocks nor log segments found. Creating new log.
Aug 11 12:04:54 dist-test-slave-mjjr krb5kdc[27266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449894, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:04:54.984196 27989 log.cc:824] T 00000000000000000000000000000000 P 0cb38209c8b04ac1a3a68103b6178202: Log is configured to *not* fsync() on all Append() calls
I20260811 12:04:54.987712 27989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cb38209c8b04ac1a3a68103b6178202: No bootstrap required, opened a new log
I20260811 12:04:54.992964 27993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:54.970431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40740 (local address 127.9.221.190:46163)
0811 12:04:54.971750 (+ 1319us) server_negotiation.cc:206] Beginning negotiation
0811 12:04:54.971762 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:04:54.971832 (+ 70us) server_negotiation.cc:407] Connection header received
0811 12:04:54.974307 (+ 2475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:04:54.974329 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:04:54.974732 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:04:54.975149 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:04:54.976607 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:54.978112 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:54.979458 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:54.980361 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:54.980558 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:54.984536 (+ 3978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:04:54.984546 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:04:54.984559 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:04:54.984599 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:04:54.988510 (+ 3911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:04:54.989191 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:04:54.989197 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:04:54.989203 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:04:54.989363 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:04:54.989764 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:04:54.989769 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:04:54.989779 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:04:54.990439 (+ 660us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:04:54.990885 (+ 446us) server_negotiation.cc:1035] Waiting for connection context
0811 12:04:54.991269 (+ 384us) server_negotiation.cc:299] Negotiation successful
0811 12:04:54.991732 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":993,"thread_start_us":623,"threads_started":1}
I20260811 12:04:54.998950 27989 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0cb38209c8b04ac1a3a68103b6178202 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cb38209c8b04ac1a3a68103b6178202" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 46163 } }
I20260811 12:04:54.999444 27989 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0cb38209c8b04ac1a3a68103b6178202 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:04:54.999596 27989 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0cb38209c8b04ac1a3a68103b6178202 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cb38209c8b04ac1a3a68103b6178202, State: Initialized, Role: FOLLOWER
I20260811 12:04:55.000311 27989 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0cb38209c8b04ac1a3a68103b6178202 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cb38209c8b04ac1a3a68103b6178202" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 46163 } }
I20260811 12:04:55.000607 27989 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0cb38209c8b04ac1a3a68103b6178202 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:04:55.000806 27989 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0cb38209c8b04ac1a3a68103b6178202 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:04:55.001031 27989 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0cb38209c8b04ac1a3a68103b6178202 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:04:55.004304 27989 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0cb38209c8b04ac1a3a68103b6178202 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cb38209c8b04ac1a3a68103b6178202" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 46163 } }
I20260811 12:04:55.004855 27989 leader_election.cc:302] T 00000000000000000000000000000000 P 0cb38209c8b04ac1a3a68103b6178202 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0cb38209c8b04ac1a3a68103b6178202; no voters:
I20260811 12:04:55.005672 27989 leader_election.cc:288] T 00000000000000000000000000000000 P 0cb38209c8b04ac1a3a68103b6178202 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:04:55.005772 27996 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0cb38209c8b04ac1a3a68103b6178202 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:04:55.007495 27996 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0cb38209c8b04ac1a3a68103b6178202 [term 1 LEADER]: Becoming Leader. State: Replica: 0cb38209c8b04ac1a3a68103b6178202, State: Running, Role: LEADER
I20260811 12:04:55.008716 27989 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0cb38209c8b04ac1a3a68103b6178202 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:04:55.008554 27996 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0cb38209c8b04ac1a3a68103b6178202 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cb38209c8b04ac1a3a68103b6178202" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 46163 } }
I20260811 12:04:55.013466 27997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cb38209c8b04ac1a3a68103b6178202 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0cb38209c8b04ac1a3a68103b6178202" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cb38209c8b04ac1a3a68103b6178202" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 46163 } } }
I20260811 12:04:55.013552 27998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cb38209c8b04ac1a3a68103b6178202 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0cb38209c8b04ac1a3a68103b6178202. Latest consensus state: current_term: 1 leader_uuid: "0cb38209c8b04ac1a3a68103b6178202" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cb38209c8b04ac1a3a68103b6178202" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 46163 } } }
I20260811 12:04:55.015679 27997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cb38209c8b04ac1a3a68103b6178202 [sys.catalog]: This master's current role is: LEADER
I20260811 12:04:55.015758 27998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cb38209c8b04ac1a3a68103b6178202 [sys.catalog]: This master's current role is: LEADER
I20260811 12:04:55.024753 28008 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:04:55.032018 28008 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:04:55.033321 27989 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 12:04:55.057631 28008 catalog_manager.cc:1347] Generated new cluster ID: f73561ea81534829846f1d473b5d6e87
I20260811 12:04:55.057817 28008 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:04:55.080175 28008 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:04:55.082369 28008 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:04:55.096866 28008 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0cb38209c8b04ac1a3a68103b6178202: Generated new TSK 0
I20260811 12:04:55.098519 28008 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.221.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab.
Entry for principal kudu/127.9.221.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab.
I20260811 12:04:56.012053 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.221.148:44357/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.129:0
--local_ip_for_outbound_sockets=127.9.221.129
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:46163
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab
--principal=kudu/127.9.221.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:39815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 12:04:56 dist-test-slave-mjjr krb5kdc[27266](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786449896, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 12:04:56.629895 28053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:04:56.630410 28053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:04:56.630630 28053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:04:56.641583 28053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:04:56.641798 28053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:04:56.642078 28053 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.129
I20260811 12:04:56.655397 28053 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:39815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.9.221.148:44357/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab
--principal=kudu/127.9.221.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:46163
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.129
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:04:56.656971 28053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:04:56.658514 28053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:04:56.678385 28067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:04:56.685315 28066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:04:56.686625 28069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:04:56.688159 28053 server_base.cc:1034] running on GCE node
I20260811 12:04:56.690615 28053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:04:56.701692 28053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:04:56.706068 28053 hybrid_clock.cc:648] HybridClock initialized: now 1786449896705996 us; error 67 us; skew 500 ppm
I20260811 12:04:56.706612 28053 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:04:56 dist-test-slave-mjjr krb5kdc[27266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449896, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:04:56.714203 28053 init.cc:377] Logged in from keytab as kudu/127.9.221.129@KRBTEST.COM (short username kudu)
I20260811 12:04:56.717329 28053 webserver.cc:460] Webserver started at http://127.9.221.129:44485/ using document root <none> and password file <none>
I20260811 12:04:56.718803 28053 fs_manager.cc:359] Metadata directory not provided
I20260811 12:04:56.719079 28053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:04:56.719682 28053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 12:04:56 dist-test-slave-mjjr krb5kdc[27266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449896, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for HTTP/127.9.221.148@KRBTEST.COM
I20260811 12:04:56.742959 28053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-0/data/instance:
uuid: "0bb2ac15cbca4eb1b5726e247f355f86"
format_stamp: "Formatted at 2026-08-11 12:04:56 on dist-test-slave-mjjr"
server_key: "36f78f24b4c6775aa406b37a5edd783f"
server_key_iv: "179c35d3b89bc70566830e1098d82f39"
server_key_version: "kuduclusterkey@0"
I20260811 12:04:56.748359 28053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance:
uuid: "0bb2ac15cbca4eb1b5726e247f355f86"
format_stamp: "Formatted at 2026-08-11 12:04:56 on dist-test-slave-mjjr"
server_key: "36f78f24b4c6775aa406b37a5edd783f"
server_key_iv: "179c35d3b89bc70566830e1098d82f39"
server_key_version: "kuduclusterkey@0"
I20260811 12:04:56.765676 28053 fs_manager.cc:691] Time spent creating directory manager: real 0.016s user 0.009s sys 0.005s
I20260811 12:04:56.800182 28080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:04:56.802532 28053 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.003s sys 0.005s
I20260811 12:04:56.802947 28053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-0/wal
uuid: "0bb2ac15cbca4eb1b5726e247f355f86"
format_stamp: "Formatted at 2026-08-11 12:04:56 on dist-test-slave-mjjr"
server_key: "36f78f24b4c6775aa406b37a5edd783f"
server_key_iv: "179c35d3b89bc70566830e1098d82f39"
server_key_version: "kuduclusterkey@0"
I20260811 12:04:56.803318 28053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:04:56.859229 28053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:04:56.866268 28053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:04:56.868515 28053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:04:56.877174 28053 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:04:56.886456 28053 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:04:56.886704 28053 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:04:56.887050 28053 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:04:56.887257 28053 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 12:04:56 dist-test-slave-mjjr krb5kdc[27266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449896, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for HTTP/127.9.221.148@KRBTEST.COM
I20260811 12:04:57.345981 28053 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.129:45343
I20260811 12:04:57.346108 28232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.129:45343 every 8 connection(s)
I20260811 12:04:57.348096 28053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
I20260811 12:04:57.352751 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 28053
I20260811 12:04:57.353045 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance
I20260811 12:04:57.354020 10102 external_mini_cluster.cc:1428] Setting key 1cdda50e9eec5d708e2c995074f75215
Aug 11 12:04:57 dist-test-slave-mjjr krb5kdc[27266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449896, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:04:57.415452 28236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:57.367958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:40381 (local address 127.9.221.190:46163)
0811 12:04:57.384606 (+ 16648us) server_negotiation.cc:206] Beginning negotiation
0811 12:04:57.384617 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:04:57.387712 (+ 3095us) server_negotiation.cc:407] Connection header received
0811 12:04:57.389589 (+ 1877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:04:57.389597 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:04:57.389700 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:04:57.389959 (+ 259us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:04:57.392564 (+ 2605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:57.393817 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:57.396466 (+ 2649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:57.397170 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:57.397304 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:57.404562 (+ 7258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:04:57.404571 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:04:57.404577 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:04:57.404626 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:04:57.410134 (+ 5508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:04:57.411428 (+ 1294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:04:57.411435 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:04:57.411440 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:04:57.411747 (+ 307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:04:57.412546 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:04:57.412552 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:04:57.412556 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:04:57.413094 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:04:57.413261 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 12:04:57.414536 (+ 1275us) server_negotiation.cc:299] Negotiation successful
0811 12:04:57.414848 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16417,"thread_start_us":16202,"threads_started":1}
I20260811 12:04:57.418570 28237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:57.367530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:46163 (local address 127.9.221.129:40381)
0811 12:04:57.386229 (+ 18699us) negotiation.cc:107] Waiting for socket to connect
0811 12:04:57.386295 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 12:04:57.389179 (+ 2884us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:04:57.390624 (+ 1445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:04:57.390638 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:04:57.391380 (+ 742us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:04:57.392328 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:04:57.392357 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:04:57.394097 (+ 1740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:04:57.394103 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:04:57.396201 (+ 2098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:04:57.396235 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:04:57.397718 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:04:57.397730 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:04:57.398055 (+ 325us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:57.399228 (+ 1173us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:04:57.399267 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:04:57.404237 (+ 4970us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:04:57.410382 (+ 6145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:04:57.410394 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:04:57.410415 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:04:57.411265 (+ 850us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:04:57.411931 (+ 666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:04:57.411938 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:04:57.411942 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:04:57.412386 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:04:57.413423 (+ 1037us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:04:57.413433 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:04:57.414004 (+ 571us) client_negotiation.cc:769] Sending connection context
0811 12:04:57.416120 (+ 2116us) client_negotiation.cc:240] Negotiation successful
0811 12:04:57.416797 (+ 677us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18324,"thread_start_us":17882,"threads_started":1}
I20260811 12:04:57.421205 28234 heartbeater.cc:347] Connected to a master server at 127.9.221.190:46163
I20260811 12:04:57.421640 28234 heartbeater.cc:464] Registering TS with master...
I20260811 12:04:57.422999 28234 heartbeater.cc:511] Master 127.9.221.190:46163 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.9.221.130@KRBTEST.COM; defaulting to no policy
I20260811 12:04:57.426011 27949 ts_manager.cc:194] Registered new tserver with Master: 0bb2ac15cbca4eb1b5726e247f355f86 (127.9.221.129:45343)
I20260811 12:04:57.428472 27949 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.129@KRBTEST.COM'} at 127.9.221.129:40381
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.130@KRBTEST.COM" created.
Aug 11 12:04:57 dist-test-slave-mjjr krb5kdc[27266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449896, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for HTTP/127.9.221.148@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab.
Entry for principal kudu/127.9.221.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab.
W20260811 12:04:57.476866 10102 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.9.221.130: real 0.119s user 0.000s sys 0.038s
I20260811 12:04:57.477669 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.221.148:44357/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.130:0
--local_ip_for_outbound_sockets=127.9.221.130
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:46163
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab
--principal=kudu/127.9.221.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:39815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:04:58.201870 28247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:04:58.202167 28247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:04:58.202322 28247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:04:58.211786 28247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:04:58.211937 28247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:04:58.212106 28247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.130
I20260811 12:04:58.223425 28247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:39815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.9.221.148:44357/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab
--principal=kudu/127.9.221.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:46163
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.130
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:04:58.224924 28247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:04:58.226271 28247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:04:58.242789 28265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:04:58.242700 28263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:04:58.243243 28262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:04:58.243846 28247 server_base.cc:1034] running on GCE node
I20260811 12:04:58.244585 28247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:04:58.245648 28247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:04:58.246896 28247 hybrid_clock.cc:648] HybridClock initialized: now 1786449898246820 us; error 47 us; skew 500 ppm
I20260811 12:04:58.247236 28247 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:04:58 dist-test-slave-mjjr krb5kdc[27266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449898, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:04:58.251144 28247 init.cc:377] Logged in from keytab as kudu/127.9.221.130@KRBTEST.COM (short username kudu)
I20260811 12:04:58.252678 28247 webserver.cc:460] Webserver started at http://127.9.221.130:43933/ using document root <none> and password file <none>
I20260811 12:04:58.253458 28247 fs_manager.cc:359] Metadata directory not provided
I20260811 12:04:58.253677 28247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:04:58.253998 28247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 12:04:58 dist-test-slave-mjjr krb5kdc[27266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449898, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.130@KRBTEST.COM for HTTP/127.9.221.148@KRBTEST.COM
I20260811 12:04:58.270205 28247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-1/data/instance:
uuid: "6d9ad35e13e3448f922dd7c2e95bc973"
format_stamp: "Formatted at 2026-08-11 12:04:58 on dist-test-slave-mjjr"
server_key: "c442654b9377a3f9bbe6a29aeca04790"
server_key_iv: "6371461419f19caa54931038af07eea6"
server_key_version: "kuduclusterkey@0"
I20260811 12:04:58.271106 28247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance:
uuid: "6d9ad35e13e3448f922dd7c2e95bc973"
format_stamp: "Formatted at 2026-08-11 12:04:58 on dist-test-slave-mjjr"
server_key: "c442654b9377a3f9bbe6a29aeca04790"
server_key_iv: "6371461419f19caa54931038af07eea6"
server_key_version: "kuduclusterkey@0"
I20260811 12:04:58.281502 28247 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.006s sys 0.006s
I20260811 12:04:58.298283 28281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:04:58.300083 28247 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260811 12:04:58.300328 28247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-1/wal
uuid: "6d9ad35e13e3448f922dd7c2e95bc973"
format_stamp: "Formatted at 2026-08-11 12:04:58 on dist-test-slave-mjjr"
server_key: "c442654b9377a3f9bbe6a29aeca04790"
server_key_iv: "6371461419f19caa54931038af07eea6"
server_key_version: "kuduclusterkey@0"
I20260811 12:04:58.300624 28247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:04:58.323557 28247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:04:58.330029 28247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:04:58.330557 28247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:04:58.332288 28247 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:04:58.334406 28247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:04:58.334553 28247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:04:58.334784 28247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:04:58.334877 28247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:04:58.418205 28247 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.130:38017
I20260811 12:04:58.418260 28428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.130:38017 every 8 connection(s)
I20260811 12:04:58.419903 28247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
I20260811 12:04:58.421192 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 28247
I20260811 12:04:58.421420 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance
I20260811 12:04:58.422191 10102 external_mini_cluster.cc:1428] Setting key ee684f61b95d89d391cc88b0c68a6dba
I20260811 12:04:58.433588 28234 heartbeater.cc:503] Master 127.9.221.190:46163 was elected leader, sending a full tablet report...
Aug 11 12:04:58 dist-test-slave-mjjr krb5kdc[27266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449898, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.130@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:04:58.447170 28433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:58.423733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:59459 (local address 127.9.221.190:46163)
0811 12:04:58.424535 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0811 12:04:58.424541 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:04:58.427736 (+ 3195us) server_negotiation.cc:407] Connection header received
0811 12:04:58.428631 (+ 895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:04:58.428636 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:04:58.428727 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:04:58.428867 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:04:58.430771 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:58.431808 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:58.433767 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:58.434441 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:58.434581 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:58.439670 (+ 5089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:04:58.439681 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:04:58.439689 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:04:58.439766 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:04:58.443320 (+ 3554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:04:58.444462 (+ 1142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:04:58.444467 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:04:58.444469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:04:58.444609 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:04:58.445188 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:04:58.445191 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:04:58.445193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:04:58.445563 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:04:58.445711 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 12:04:58.446683 (+ 972us) server_negotiation.cc:299] Negotiation successful
0811 12:04:58.446844 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":473,"threads_started":1}
I20260811 12:04:58.449103 28434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:58.424575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:46163 (local address 127.9.221.130:59459)
0811 12:04:58.425959 (+ 1384us) negotiation.cc:107] Waiting for socket to connect
0811 12:04:58.426005 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 12:04:58.428369 (+ 2364us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:04:58.429155 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:04:58.429162 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:04:58.429792 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:04:58.430601 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:04:58.430620 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:04:58.431995 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:04:58.431999 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:04:58.433585 (+ 1586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:04:58.433600 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:04:58.434937 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:04:58.434946 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:04:58.435352 (+ 406us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:58.436514 (+ 1162us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:04:58.436544 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:04:58.439310 (+ 2766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:04:58.443469 (+ 4159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:04:58.443481 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:04:58.443500 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:04:58.444349 (+ 849us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:04:58.444723 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:04:58.444731 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:04:58.444737 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:04:58.445046 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:04:58.445701 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:04:58.445712 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:04:58.446221 (+ 509us) client_negotiation.cc:769] Sending connection context
0811 12:04:58.446578 (+ 357us) client_negotiation.cc:240] Negotiation successful
0811 12:04:58.447346 (+ 768us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1049,"thread_start_us":648,"threads_started":1}
I20260811 12:04:58.451001 28430 heartbeater.cc:347] Connected to a master server at 127.9.221.190:46163
I20260811 12:04:58.451399 28430 heartbeater.cc:464] Registering TS with master...
I20260811 12:04:58.452580 28430 heartbeater.cc:511] Master 127.9.221.190:46163 requested a full tablet report, sending...
I20260811 12:04:58.454710 27949 ts_manager.cc:194] Registered new tserver with Master: 6d9ad35e13e3448f922dd7c2e95bc973 (127.9.221.130:38017)
I20260811 12:04:58.455739 27949 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.130@KRBTEST.COM'} at 127.9.221.130:59459
WARNING: no policy specified for kudu/127.9.221.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab.
Entry for principal kudu/127.9.221.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab.
I20260811 12:04:58.492605 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.221.148:44357/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.131:0
--local_ip_for_outbound_sockets=127.9.221.131
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:46163
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab
--principal=kudu/127.9.221.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:39815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:04:59.031056 28438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:04:59.031500 28438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:04:59.031772 28438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:04:59.041927 28438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:04:59.042171 28438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:04:59.042413 28438 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.131
I20260811 12:04:59.054872 28438 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:39815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.9.221.148:44357/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab
--principal=kudu/127.9.221.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:46163
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.131
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:04:59.056579 28438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:04:59.058236 28438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:04:59.078856 28448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:04:59.079157 28455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:04:59.078866 28451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:04:59.080852 28438 server_base.cc:1034] running on GCE node
I20260811 12:04:59.081524 28438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:04:59.082830 28438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:04:59.084118 28438 hybrid_clock.cc:648] HybridClock initialized: now 1786449899084015 us; error 38 us; skew 500 ppm
I20260811 12:04:59.084491 28438 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:04:59 dist-test-slave-mjjr krb5kdc[27266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449899, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:04:59.088897 28438 init.cc:377] Logged in from keytab as kudu/127.9.221.131@KRBTEST.COM (short username kudu)
I20260811 12:04:59.090667 28438 webserver.cc:460] Webserver started at http://127.9.221.131:38291/ using document root <none> and password file <none>
I20260811 12:04:59.091495 28438 fs_manager.cc:359] Metadata directory not provided
I20260811 12:04:59.091735 28438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:04:59.092239 28438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 12:04:59 dist-test-slave-mjjr krb5kdc[27266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449899, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.131@KRBTEST.COM for HTTP/127.9.221.148@KRBTEST.COM
I20260811 12:04:59.110762 28438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-2/data/instance:
uuid: "d65dabfa42a14db4af98d042423ec48f"
format_stamp: "Formatted at 2026-08-11 12:04:59 on dist-test-slave-mjjr"
server_key: "be4970028aaafffab1acc2b6330abb31"
server_key_iv: "bf98b02949a859cc39bb21c76935884b"
server_key_version: "kuduclusterkey@0"
I20260811 12:04:59.111881 28438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance:
uuid: "d65dabfa42a14db4af98d042423ec48f"
format_stamp: "Formatted at 2026-08-11 12:04:59 on dist-test-slave-mjjr"
server_key: "be4970028aaafffab1acc2b6330abb31"
server_key_iv: "bf98b02949a859cc39bb21c76935884b"
server_key_version: "kuduclusterkey@0"
I20260811 12:04:59.118808 28438 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260811 12:04:59.135987 28467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:04:59.137676 28438 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260811 12:04:59.137913 28438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-2/wal
uuid: "d65dabfa42a14db4af98d042423ec48f"
format_stamp: "Formatted at 2026-08-11 12:04:59 on dist-test-slave-mjjr"
server_key: "be4970028aaafffab1acc2b6330abb31"
server_key_iv: "bf98b02949a859cc39bb21c76935884b"
server_key_version: "kuduclusterkey@0"
I20260811 12:04:59.138175 28438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:04:59.166399 28438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:04:59.171569 28438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:04:59.172062 28438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:04:59.173584 28438 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:04:59.175781 28438 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:04:59.176002 28438 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260811 12:04:59.176220 28438 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:04:59.176347 28438 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:04:59.254284 28438 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.131:44667
I20260811 12:04:59.254278 28628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.131:44667 every 8 connection(s)
I20260811 12:04:59.256013 28438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
I20260811 12:04:59.264204 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 28438
I20260811 12:04:59.264400 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance
I20260811 12:04:59.265230 10102 external_mini_cluster.cc:1428] Setting key 94635a28a080d5d09b86e89c1920911b
Aug 11 12:04:59 dist-test-slave-mjjr krb5kdc[27266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449899, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.131@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:04:59.282367 28631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:59.259470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:42103 (local address 127.9.221.190:46163)
0811 12:04:59.260656 (+ 1186us) server_negotiation.cc:206] Beginning negotiation
0811 12:04:59.260665 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:04:59.261659 (+ 994us) server_negotiation.cc:407] Connection header received
0811 12:04:59.264032 (+ 2373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:04:59.264037 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:04:59.264112 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:04:59.264217 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:04:59.266820 (+ 2603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:59.268074 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:59.270083 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:59.270613 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:59.270721 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:59.274859 (+ 4138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:04:59.274865 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:04:59.274870 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:04:59.274905 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:04:59.278375 (+ 3470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:04:59.279243 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:04:59.279249 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:04:59.279251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:04:59.279438 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:04:59.280210 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:04:59.280215 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:04:59.280219 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:04:59.280624 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:04:59.280869 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0811 12:04:59.281615 (+ 746us) server_negotiation.cc:299] Negotiation successful
0811 12:04:59.281883 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1049,"thread_start_us":818,"threads_started":1}
I20260811 12:04:59.283696 28632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:59.260086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:46163 (local address 127.9.221.131:42103)
0811 12:04:59.261493 (+ 1407us) negotiation.cc:107] Waiting for socket to connect
0811 12:04:59.261533 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 12:04:59.263689 (+ 2156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:04:59.264544 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:04:59.264556 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:04:59.265513 (+ 957us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:04:59.266638 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:04:59.266660 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:04:59.268260 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:04:59.268265 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:04:59.269925 (+ 1660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:04:59.269937 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:04:59.270873 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:04:59.270878 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:04:59.271149 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:59.271900 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:04:59.271926 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:04:59.274604 (+ 2678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:04:59.278529 (+ 3925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:04:59.278538 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:04:59.278553 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:04:59.279122 (+ 569us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:04:59.279551 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:04:59.279555 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:04:59.279557 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:04:59.279945 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:04:59.280805 (+ 860us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:04:59.280814 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:04:59.281208 (+ 394us) client_negotiation.cc:769] Sending connection context
0811 12:04:59.281534 (+ 326us) client_negotiation.cc:240] Negotiation successful
0811 12:04:59.282141 (+ 607us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1080,"thread_start_us":695,"threads_started":1}
I20260811 12:04:59.285187 28629 heartbeater.cc:347] Connected to a master server at 127.9.221.190:46163
I20260811 12:04:59.285531 28629 heartbeater.cc:464] Registering TS with master...
I20260811 12:04:59.286405 28629 heartbeater.cc:511] Master 127.9.221.190:46163 requested a full tablet report, sending...
I20260811 12:04:59.288215 27949 ts_manager.cc:194] Registered new tserver with Master: d65dabfa42a14db4af98d042423ec48f (127.9.221.131:44667)
I20260811 12:04:59.289254 27949 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.131@KRBTEST.COM'} at 127.9.221.131:42103
I20260811 12:04:59.294329 10102 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:04:59.316661 28631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:59.300694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37434 (local address 127.9.221.190:46163)
0811 12:04:59.300903 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 12:04:59.300917 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 12:04:59.301760 (+ 843us) server_negotiation.cc:407] Connection header received
0811 12:04:59.301891 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:04:59.301895 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:04:59.301968 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:04:59.302049 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:04:59.303923 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:59.304745 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:59.306485 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:59.306955 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:59.307060 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:59.310437 (+ 3377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:04:59.310448 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:04:59.310451 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:04:59.310483 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:04:59.313855 (+ 3372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:04:59.314623 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:04:59.314627 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:04:59.314630 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:04:59.314788 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:04:59.315197 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:04:59.315200 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:04:59.315202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:04:59.315491 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:04:59.315772 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0811 12:04:59.316092 (+ 320us) server_negotiation.cc:299] Negotiation successful
0811 12:04:59.316286 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 12:04:59.322062 27949 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37434:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:04:59.334533 27949 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:04:59.378068 28650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:59.351792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52050 (local address 127.9.221.131:44667)
0811 12:04:59.363011 (+ 11219us) server_negotiation.cc:206] Beginning negotiation
0811 12:04:59.363027 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 12:04:59.364623 (+ 1596us) server_negotiation.cc:407] Connection header received
0811 12:04:59.364798 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:04:59.364807 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:04:59.365138 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:04:59.366582 (+ 1444us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:04:59.367842 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:59.370050 (+ 2208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:59.374489 (+ 4439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:59.376531 (+ 2042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:59.376686 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:59.377029 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0811 12:04:59.377343 (+ 314us) server_negotiation.cc:299] Negotiation successful
0811 12:04:59.377521 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10991,"thread_start_us":10769,"threads_started":1}
I20260811 12:04:59.378124 28648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:59.351222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.131:44667 (local address 127.0.0.1:52050)
0811 12:04:59.364002 (+ 12780us) negotiation.cc:107] Waiting for socket to connect
0811 12:04:59.364073 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0811 12:04:59.364353 (+ 280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:04:59.365439 (+ 1086us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:04:59.365449 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:04:59.365481 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:04:59.367378 (+ 1897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:04:59.367394 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:04:59.370278 (+ 2884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:04:59.370299 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:04:59.374318 (+ 4019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:04:59.374331 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:04:59.376873 (+ 2542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:04:59.376881 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:04:59.377091 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:59.377137 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 12:04:59.377287 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 12:04:59.377482 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12323,"thread_start_us":12054,"threads_started":1}
I20260811 12:04:59.381347 28651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:59.351996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:45343 (local address 127.0.0.1:42986)
0811 12:04:59.365367 (+ 13371us) negotiation.cc:107] Waiting for socket to connect
0811 12:04:59.365387 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:04:59.365549 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:04:59.367962 (+ 2413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:04:59.367968 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:04:59.368007 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:04:59.370227 (+ 2220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:04:59.370244 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:04:59.373517 (+ 3273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:04:59.373531 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:04:59.376872 (+ 3341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:04:59.376884 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:04:59.380302 (+ 3418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:04:59.380311 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:04:59.380529 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:59.380574 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 12:04:59.380671 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 12:04:59.380803 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13164,"spinlock_wait_cycles":2304,"thread_start_us":13647,"threads_started":1}
I20260811 12:04:59.381587 28652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:59.352048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42986 (local address 127.9.221.129:45343)
0811 12:04:59.366572 (+ 14524us) server_negotiation.cc:206] Beginning negotiation
0811 12:04:59.366580 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:04:59.366610 (+ 30us) server_negotiation.cc:407] Connection header received
0811 12:04:59.366770 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:04:59.366780 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:04:59.367017 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:04:59.367394 (+ 377us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:04:59.370574 (+ 3180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:59.373004 (+ 2430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:59.377079 (+ 4075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:59.379588 (+ 2509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:59.379810 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:59.380092 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0811 12:04:59.380800 (+ 708us) server_negotiation.cc:299] Negotiation successful
0811 12:04:59.380962 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14312,"thread_start_us":14140,"threads_started":1}
I20260811 12:04:59.383405 28539 tablet_service.cc:1467] Processing CreateTablet for tablet e1a632c9758343a6890d4bfe6aa383d5 (DEFAULT_TABLE table=test-table [id=7ecb8fe4af424e14b529ed6247fafd03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:04:59.385231 28653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:59.366193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.130:38017 (local address 127.0.0.1:49716)
0811 12:04:59.374605 (+ 8412us) negotiation.cc:107] Waiting for socket to connect
0811 12:04:59.374625 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:04:59.374786 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:04:59.375732 (+ 946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:04:59.375737 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:04:59.375762 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:04:59.376584 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:04:59.376598 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:04:59.379268 (+ 2670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:04:59.379273 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:04:59.381896 (+ 2623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:04:59.381915 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:04:59.384193 (+ 2278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:04:59.384201 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:04:59.384439 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:59.384473 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 12:04:59.384577 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 12:04:59.384702 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8234,"mutex_wait_us":1,"thread_start_us":8716,"threads_started":1}
I20260811 12:04:59.385800 28539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1a632c9758343a6890d4bfe6aa383d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:04:59.385337 28143 tablet_service.cc:1467] Processing CreateTablet for tablet e1a632c9758343a6890d4bfe6aa383d5 (DEFAULT_TABLE table=test-table [id=7ecb8fe4af424e14b529ed6247fafd03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:04:59.386282 28656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:59.366521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49716 (local address 127.9.221.130:38017)
0811 12:04:59.370686 (+ 4165us) server_negotiation.cc:206] Beginning negotiation
0811 12:04:59.370691 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:04:59.374748 (+ 4057us) server_negotiation.cc:407] Connection header received
0811 12:04:59.374971 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:04:59.374979 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:04:59.375304 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:04:59.375754 (+ 450us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:04:59.376861 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:59.379054 (+ 2193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:59.382108 (+ 3054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:59.383870 (+ 1762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:59.384011 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:59.384184 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 12:04:59.384639 (+ 455us) server_negotiation.cc:299] Negotiation successful
0811 12:04:59.385656 (+ 1017us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3999,"thread_start_us":3775,"threads_started":1}
I20260811 12:04:59.387110 28143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1a632c9758343a6890d4bfe6aa383d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:04:59.390329 28342 tablet_service.cc:1467] Processing CreateTablet for tablet e1a632c9758343a6890d4bfe6aa383d5 (DEFAULT_TABLE table=test-table [id=7ecb8fe4af424e14b529ed6247fafd03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:04:59.391968 28342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1a632c9758343a6890d4bfe6aa383d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:04:59.401407 28659 tablet_bootstrap.cc:492] T e1a632c9758343a6890d4bfe6aa383d5 P 0bb2ac15cbca4eb1b5726e247f355f86: Bootstrap starting.
I20260811 12:04:59.407217 28659 tablet_bootstrap.cc:654] T e1a632c9758343a6890d4bfe6aa383d5 P 0bb2ac15cbca4eb1b5726e247f355f86: Neither blocks nor log segments found. Creating new log.
I20260811 12:04:59.408831 28659 log.cc:824] T e1a632c9758343a6890d4bfe6aa383d5 P 0bb2ac15cbca4eb1b5726e247f355f86: Log is configured to *not* fsync() on all Append() calls
I20260811 12:04:59.413339 28662 tablet_bootstrap.cc:492] T e1a632c9758343a6890d4bfe6aa383d5 P d65dabfa42a14db4af98d042423ec48f: Bootstrap starting.
I20260811 12:04:59.415233 28659 tablet_bootstrap.cc:492] T e1a632c9758343a6890d4bfe6aa383d5 P 0bb2ac15cbca4eb1b5726e247f355f86: No bootstrap required, opened a new log
I20260811 12:04:59.415546 28659 ts_tablet_manager.cc:1397] T e1a632c9758343a6890d4bfe6aa383d5 P 0bb2ac15cbca4eb1b5726e247f355f86: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.004s
I20260811 12:04:59.418303 28662 tablet_bootstrap.cc:654] T e1a632c9758343a6890d4bfe6aa383d5 P d65dabfa42a14db4af98d042423ec48f: Neither blocks nor log segments found. Creating new log.
I20260811 12:04:59.420138 28662 log.cc:824] T e1a632c9758343a6890d4bfe6aa383d5 P d65dabfa42a14db4af98d042423ec48f: Log is configured to *not* fsync() on all Append() calls
I20260811 12:04:59.421520 28665 tablet_bootstrap.cc:492] T e1a632c9758343a6890d4bfe6aa383d5 P 6d9ad35e13e3448f922dd7c2e95bc973: Bootstrap starting.
I20260811 12:04:59.426890 28665 tablet_bootstrap.cc:654] T e1a632c9758343a6890d4bfe6aa383d5 P 6d9ad35e13e3448f922dd7c2e95bc973: Neither blocks nor log segments found. Creating new log.
I20260811 12:04:59.429100 28665 log.cc:824] T e1a632c9758343a6890d4bfe6aa383d5 P 6d9ad35e13e3448f922dd7c2e95bc973: Log is configured to *not* fsync() on all Append() calls
I20260811 12:04:59.432397 28659 raft_consensus.cc:348] T e1a632c9758343a6890d4bfe6aa383d5 P 0bb2ac15cbca4eb1b5726e247f355f86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d65dabfa42a14db4af98d042423ec48f" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 44667 } } peers { permanent_uuid: "0bb2ac15cbca4eb1b5726e247f355f86" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 45343 } } peers { permanent_uuid: "6d9ad35e13e3448f922dd7c2e95bc973" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38017 } }
I20260811 12:04:59.433106 28659 raft_consensus.cc:374] T e1a632c9758343a6890d4bfe6aa383d5 P 0bb2ac15cbca4eb1b5726e247f355f86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:04:59.433337 28659 raft_consensus.cc:729] T e1a632c9758343a6890d4bfe6aa383d5 P 0bb2ac15cbca4eb1b5726e247f355f86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bb2ac15cbca4eb1b5726e247f355f86, State: Initialized, Role: FOLLOWER
I20260811 12:04:59.434376 28659 consensus_queue.cc:260] T e1a632c9758343a6890d4bfe6aa383d5 P 0bb2ac15cbca4eb1b5726e247f355f86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d65dabfa42a14db4af98d042423ec48f" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 44667 } } peers { permanent_uuid: "0bb2ac15cbca4eb1b5726e247f355f86" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 45343 } } peers { permanent_uuid: "6d9ad35e13e3448f922dd7c2e95bc973" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38017 } }
I20260811 12:04:59.438159 28662 tablet_bootstrap.cc:492] T e1a632c9758343a6890d4bfe6aa383d5 P d65dabfa42a14db4af98d042423ec48f: No bootstrap required, opened a new log
I20260811 12:04:59.438560 28662 ts_tablet_manager.cc:1397] T e1a632c9758343a6890d4bfe6aa383d5 P d65dabfa42a14db4af98d042423ec48f: Time spent bootstrapping tablet: real 0.025s user 0.008s sys 0.013s
I20260811 12:04:59.446017 28659 ts_tablet_manager.cc:1428] T e1a632c9758343a6890d4bfe6aa383d5 P 0bb2ac15cbca4eb1b5726e247f355f86: Time spent starting tablet: real 0.030s user 0.017s sys 0.010s
I20260811 12:04:59.447068 28665 tablet_bootstrap.cc:492] T e1a632c9758343a6890d4bfe6aa383d5 P 6d9ad35e13e3448f922dd7c2e95bc973: No bootstrap required, opened a new log
I20260811 12:04:59.447419 28665 ts_tablet_manager.cc:1397] T e1a632c9758343a6890d4bfe6aa383d5 P 6d9ad35e13e3448f922dd7c2e95bc973: Time spent bootstrapping tablet: real 0.028s user 0.000s sys 0.021s
I20260811 12:04:59.451903 28662 raft_consensus.cc:348] T e1a632c9758343a6890d4bfe6aa383d5 P d65dabfa42a14db4af98d042423ec48f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d65dabfa42a14db4af98d042423ec48f" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 44667 } } peers { permanent_uuid: "0bb2ac15cbca4eb1b5726e247f355f86" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 45343 } } peers { permanent_uuid: "6d9ad35e13e3448f922dd7c2e95bc973" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38017 } }
I20260811 12:04:59.452332 28662 raft_consensus.cc:374] T e1a632c9758343a6890d4bfe6aa383d5 P d65dabfa42a14db4af98d042423ec48f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:04:59.452474 28662 raft_consensus.cc:729] T e1a632c9758343a6890d4bfe6aa383d5 P d65dabfa42a14db4af98d042423ec48f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d65dabfa42a14db4af98d042423ec48f, State: Initialized, Role: FOLLOWER
I20260811 12:04:59.453024 28662 consensus_queue.cc:260] T e1a632c9758343a6890d4bfe6aa383d5 P d65dabfa42a14db4af98d042423ec48f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d65dabfa42a14db4af98d042423ec48f" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 44667 } } peers { permanent_uuid: "0bb2ac15cbca4eb1b5726e247f355f86" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 45343 } } peers { permanent_uuid: "6d9ad35e13e3448f922dd7c2e95bc973" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38017 } }
I20260811 12:04:59.454466 28629 heartbeater.cc:503] Master 127.9.221.190:46163 was elected leader, sending a full tablet report...
I20260811 12:04:59.455022 28662 ts_tablet_manager.cc:1428] T e1a632c9758343a6890d4bfe6aa383d5 P d65dabfa42a14db4af98d042423ec48f: Time spent starting tablet: real 0.016s user 0.013s sys 0.004s
I20260811 12:04:59.458815 28665 raft_consensus.cc:348] T e1a632c9758343a6890d4bfe6aa383d5 P 6d9ad35e13e3448f922dd7c2e95bc973 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d65dabfa42a14db4af98d042423ec48f" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 44667 } } peers { permanent_uuid: "0bb2ac15cbca4eb1b5726e247f355f86" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 45343 } } peers { permanent_uuid: "6d9ad35e13e3448f922dd7c2e95bc973" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38017 } }
I20260811 12:04:59.459427 28665 raft_consensus.cc:374] T e1a632c9758343a6890d4bfe6aa383d5 P 6d9ad35e13e3448f922dd7c2e95bc973 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:04:59.459510 28430 heartbeater.cc:503] Master 127.9.221.190:46163 was elected leader, sending a full tablet report...
I20260811 12:04:59.459589 28665 raft_consensus.cc:729] T e1a632c9758343a6890d4bfe6aa383d5 P 6d9ad35e13e3448f922dd7c2e95bc973 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d9ad35e13e3448f922dd7c2e95bc973, State: Initialized, Role: FOLLOWER
I20260811 12:04:59.460265 28665 consensus_queue.cc:260] T e1a632c9758343a6890d4bfe6aa383d5 P 6d9ad35e13e3448f922dd7c2e95bc973 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d65dabfa42a14db4af98d042423ec48f" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 44667 } } peers { permanent_uuid: "0bb2ac15cbca4eb1b5726e247f355f86" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 45343 } } peers { permanent_uuid: "6d9ad35e13e3448f922dd7c2e95bc973" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38017 } }
I20260811 12:04:59.462452 28665 ts_tablet_manager.cc:1428] T e1a632c9758343a6890d4bfe6aa383d5 P 6d9ad35e13e3448f922dd7c2e95bc973: Time spent starting tablet: real 0.015s user 0.013s sys 0.003s
I20260811 12:04:59.504426 28673 raft_consensus.cc:482] T e1a632c9758343a6890d4bfe6aa383d5 P 6d9ad35e13e3448f922dd7c2e95bc973 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:04:59.504854 28673 raft_consensus.cc:504] T e1a632c9758343a6890d4bfe6aa383d5 P 6d9ad35e13e3448f922dd7c2e95bc973 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d65dabfa42a14db4af98d042423ec48f" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 44667 } } peers { permanent_uuid: "0bb2ac15cbca4eb1b5726e247f355f86" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 45343 } } peers { permanent_uuid: "6d9ad35e13e3448f922dd7c2e95bc973" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38017 } }
I20260811 12:04:59.507150 28673 leader_election.cc:288] T e1a632c9758343a6890d4bfe6aa383d5 P 6d9ad35e13e3448f922dd7c2e95bc973 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d65dabfa42a14db4af98d042423ec48f (127.9.221.131:44667), 0bb2ac15cbca4eb1b5726e247f355f86 (127.9.221.129:45343)
W20260811 12:04:59.508726 28630 tablet.cc:2369] T e1a632c9758343a6890d4bfe6aa383d5 P d65dabfa42a14db4af98d042423ec48f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:04:59.518923 28652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:59.507470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:46443 (local address 127.9.221.129:45343)
0811 12:04:59.507780 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0811 12:04:59.507789 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:04:59.509931 (+ 2142us) server_negotiation.cc:407] Connection header received
0811 12:04:59.510193 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:04:59.510217 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:04:59.510310 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:04:59.510408 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:04:59.511259 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:59.512577 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:59.515956 (+ 3379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:59.517675 (+ 1719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:59.517873 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:59.517951 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 12:04:59.518444 (+ 493us) server_negotiation.cc:299] Negotiation successful
0811 12:04:59.518584 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260811 12:04:59.519240 28650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:59.507404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:46045 (local address 127.9.221.131:44667)
0811 12:04:59.507770 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0811 12:04:59.507779 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:04:59.508972 (+ 1193us) server_negotiation.cc:407] Connection header received
0811 12:04:59.509374 (+ 402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:04:59.509402 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:04:59.509497 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:04:59.509570 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:04:59.510963 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:59.512309 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:59.516004 (+ 3695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:59.517681 (+ 1677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:59.517876 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:59.517955 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 12:04:59.518761 (+ 806us) server_negotiation.cc:299] Negotiation successful
0811 12:04:59.518924 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 12:04:59.519529 28676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:59.508514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:45343 (local address 127.9.221.130:46443)
0811 12:04:59.509813 (+ 1299us) negotiation.cc:107] Waiting for socket to connect
0811 12:04:59.509837 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 12:04:59.510028 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:04:59.510436 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:04:59.510440 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:04:59.510465 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:04:59.511035 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:04:59.511045 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:04:59.512773 (+ 1728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:04:59.512779 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:04:59.515766 (+ 2987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:04:59.515792 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:04:59.517934 (+ 2142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:04:59.517942 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:04:59.518226 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:59.518261 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 12:04:59.518372 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 12:04:59.518469 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1123,"mutex_wait_us":53,"thread_start_us":872,"threads_started":1}
I20260811 12:04:59.519599 28675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:59.507439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.131:44667 (local address 127.9.221.130:46045)
0811 12:04:59.508787 (+ 1348us) negotiation.cc:107] Waiting for socket to connect
0811 12:04:59.508810 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:04:59.509166 (+ 356us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:04:59.509909 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:04:59.509919 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:04:59.509943 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:04:59.510791 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:04:59.510807 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:04:59.512435 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:04:59.512439 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:04:59.515837 (+ 3398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:04:59.515851 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:04:59.518314 (+ 2463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:04:59.518320 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:04:59.518569 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:59.518605 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 12:04:59.518701 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 12:04:59.518815 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1173,"mutex_wait_us":112,"thread_start_us":855,"threads_started":1}
I20260811 12:04:59.520936 28564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1a632c9758343a6890d4bfe6aa383d5" candidate_uuid: "6d9ad35e13e3448f922dd7c2e95bc973" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d65dabfa42a14db4af98d042423ec48f" is_pre_election: true
I20260811 12:04:59.520955 28167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1a632c9758343a6890d4bfe6aa383d5" candidate_uuid: "6d9ad35e13e3448f922dd7c2e95bc973" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bb2ac15cbca4eb1b5726e247f355f86" is_pre_election: true
I20260811 12:04:59.521544 28564 raft_consensus.cc:2454] T e1a632c9758343a6890d4bfe6aa383d5 P d65dabfa42a14db4af98d042423ec48f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d9ad35e13e3448f922dd7c2e95bc973 in term 0.
I20260811 12:04:59.521639 28167 raft_consensus.cc:2454] T e1a632c9758343a6890d4bfe6aa383d5 P 0bb2ac15cbca4eb1b5726e247f355f86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d9ad35e13e3448f922dd7c2e95bc973 in term 0.
I20260811 12:04:59.522522 28283 leader_election.cc:302] T e1a632c9758343a6890d4bfe6aa383d5 P 6d9ad35e13e3448f922dd7c2e95bc973 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0bb2ac15cbca4eb1b5726e247f355f86, 6d9ad35e13e3448f922dd7c2e95bc973, d65dabfa42a14db4af98d042423ec48f; no voters:
I20260811 12:04:59.523195 28673 raft_consensus.cc:2781] T e1a632c9758343a6890d4bfe6aa383d5 P 6d9ad35e13e3448f922dd7c2e95bc973 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:04:59.523370 28673 raft_consensus.cc:482] T e1a632c9758343a6890d4bfe6aa383d5 P 6d9ad35e13e3448f922dd7c2e95bc973 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:04:59.523523 28673 raft_consensus.cc:3037] T e1a632c9758343a6890d4bfe6aa383d5 P 6d9ad35e13e3448f922dd7c2e95bc973 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:04:59.526546 28673 raft_consensus.cc:504] T e1a632c9758343a6890d4bfe6aa383d5 P 6d9ad35e13e3448f922dd7c2e95bc973 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d65dabfa42a14db4af98d042423ec48f" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 44667 } } peers { permanent_uuid: "0bb2ac15cbca4eb1b5726e247f355f86" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 45343 } } peers { permanent_uuid: "6d9ad35e13e3448f922dd7c2e95bc973" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38017 } }
I20260811 12:04:59.527336 28673 leader_election.cc:288] T e1a632c9758343a6890d4bfe6aa383d5 P 6d9ad35e13e3448f922dd7c2e95bc973 [CANDIDATE]: Term 1 election: Requested vote from peers d65dabfa42a14db4af98d042423ec48f (127.9.221.131:44667), 0bb2ac15cbca4eb1b5726e247f355f86 (127.9.221.129:45343)
I20260811 12:04:59.527944 28167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1a632c9758343a6890d4bfe6aa383d5" candidate_uuid: "6d9ad35e13e3448f922dd7c2e95bc973" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bb2ac15cbca4eb1b5726e247f355f86"
I20260811 12:04:59.527952 28564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1a632c9758343a6890d4bfe6aa383d5" candidate_uuid: "6d9ad35e13e3448f922dd7c2e95bc973" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d65dabfa42a14db4af98d042423ec48f"
I20260811 12:04:59.528192 28564 raft_consensus.cc:3037] T e1a632c9758343a6890d4bfe6aa383d5 P d65dabfa42a14db4af98d042423ec48f [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:04:59.528213 28167 raft_consensus.cc:3037] T e1a632c9758343a6890d4bfe6aa383d5 P 0bb2ac15cbca4eb1b5726e247f355f86 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:04:59.530979 28564 raft_consensus.cc:2454] T e1a632c9758343a6890d4bfe6aa383d5 P d65dabfa42a14db4af98d042423ec48f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d9ad35e13e3448f922dd7c2e95bc973 in term 1.
I20260811 12:04:59.531081 28167 raft_consensus.cc:2454] T e1a632c9758343a6890d4bfe6aa383d5 P 0bb2ac15cbca4eb1b5726e247f355f86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d9ad35e13e3448f922dd7c2e95bc973 in term 1.
I20260811 12:04:59.531495 28283 leader_election.cc:302] T e1a632c9758343a6890d4bfe6aa383d5 P 6d9ad35e13e3448f922dd7c2e95bc973 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d9ad35e13e3448f922dd7c2e95bc973, d65dabfa42a14db4af98d042423ec48f; no voters:
I20260811 12:04:59.531852 28673 raft_consensus.cc:2781] T e1a632c9758343a6890d4bfe6aa383d5 P 6d9ad35e13e3448f922dd7c2e95bc973 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:04:59.532596 28673 raft_consensus.cc:686] T e1a632c9758343a6890d4bfe6aa383d5 P 6d9ad35e13e3448f922dd7c2e95bc973 [term 1 LEADER]: Becoming Leader. State: Replica: 6d9ad35e13e3448f922dd7c2e95bc973, State: Running, Role: LEADER
I20260811 12:04:59.533300 28673 consensus_queue.cc:237] T e1a632c9758343a6890d4bfe6aa383d5 P 6d9ad35e13e3448f922dd7c2e95bc973 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d65dabfa42a14db4af98d042423ec48f" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 44667 } } peers { permanent_uuid: "0bb2ac15cbca4eb1b5726e247f355f86" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 45343 } } peers { permanent_uuid: "6d9ad35e13e3448f922dd7c2e95bc973" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38017 } }
I20260811 12:04:59.539058 27949 catalog_manager.cc:5495] T e1a632c9758343a6890d4bfe6aa383d5 P 6d9ad35e13e3448f922dd7c2e95bc973 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d9ad35e13e3448f922dd7c2e95bc973 (127.9.221.130). New cstate: current_term: 1 leader_uuid: "6d9ad35e13e3448f922dd7c2e95bc973" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d65dabfa42a14db4af98d042423ec48f" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 44667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bb2ac15cbca4eb1b5726e247f355f86" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 45343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d9ad35e13e3448f922dd7c2e95bc973" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 38017 } health_report { overall_health: HEALTHY } } }
I20260811 12:04:59.570534 28656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:59.563328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49732 (local address 127.9.221.130:38017)
0811 12:04:59.563592 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 12:04:59.563599 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:04:59.563624 (+ 25us) server_negotiation.cc:407] Connection header received
0811 12:04:59.563762 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:04:59.563766 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:04:59.563845 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:04:59.564012 (+ 167us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:04:59.564540 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:59.565447 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:59.567782 (+ 2335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:59.568356 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:59.568532 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:59.569245 (+ 713us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:04:59.569792 (+ 547us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:04:59.569954 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 12:04:59.570041 (+ 87us) server_negotiation.cc:299] Negotiation successful
0811 12:04:59.570127 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 12:04:59.580188 28564 raft_consensus.cc:1261] T e1a632c9758343a6890d4bfe6aa383d5 P d65dabfa42a14db4af98d042423ec48f [term 1 FOLLOWER]: Refusing update from remote peer 6d9ad35e13e3448f922dd7c2e95bc973: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:04:59.580235 28167 raft_consensus.cc:1261] T e1a632c9758343a6890d4bfe6aa383d5 P 0bb2ac15cbca4eb1b5726e247f355f86 [term 1 FOLLOWER]: Refusing update from remote peer 6d9ad35e13e3448f922dd7c2e95bc973: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:04:59.581441 28679 consensus_queue.cc:1035] T e1a632c9758343a6890d4bfe6aa383d5 P 6d9ad35e13e3448f922dd7c2e95bc973 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bb2ac15cbca4eb1b5726e247f355f86" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 45343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:04:59.582031 28673 consensus_queue.cc:1035] T e1a632c9758343a6890d4bfe6aa383d5 P 6d9ad35e13e3448f922dd7c2e95bc973 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d65dabfa42a14db4af98d042423ec48f" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 44667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:04:59.618371 27948 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37434:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:04:59.618849 27948 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37434:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:04:59.623687 27948 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0cb38209c8b04ac1a3a68103b6178202: Sending DeleteTablet for 3 replicas of tablet e1a632c9758343a6890d4bfe6aa383d5
I20260811 12:04:59.625113 28539 tablet_service.cc:1514] Processing DeleteTablet for tablet e1a632c9758343a6890d4bfe6aa383d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:04:59 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:52050
I20260811 12:04:59.625599 28143 tablet_service.cc:1514] Processing DeleteTablet for tablet e1a632c9758343a6890d4bfe6aa383d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:04:59 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:42986
I20260811 12:04:59.626730 28342 tablet_service.cc:1514] Processing DeleteTablet for tablet e1a632c9758343a6890d4bfe6aa383d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:04:59 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:49716
I20260811 12:04:59.628027 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 28053
I20260811 12:04:59.639171 28700 tablet_replica.cc:330] T e1a632c9758343a6890d4bfe6aa383d5 P 0bb2ac15cbca4eb1b5726e247f355f86: stopping tablet replica
I20260811 12:04:59.639946 28700 raft_consensus.cc:2229] T e1a632c9758343a6890d4bfe6aa383d5 P 0bb2ac15cbca4eb1b5726e247f355f86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:04:59.640923 28700 raft_consensus.cc:2258] T e1a632c9758343a6890d4bfe6aa383d5 P 0bb2ac15cbca4eb1b5726e247f355f86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:04:59.644202 28700 ts_tablet_manager.cc:1905] T e1a632c9758343a6890d4bfe6aa383d5 P 0bb2ac15cbca4eb1b5726e247f355f86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:04:59.648056 28701 tablet_replica.cc:330] T e1a632c9758343a6890d4bfe6aa383d5 P 6d9ad35e13e3448f922dd7c2e95bc973: stopping tablet replica
I20260811 12:04:59.648627 28701 raft_consensus.cc:2229] T e1a632c9758343a6890d4bfe6aa383d5 P 6d9ad35e13e3448f922dd7c2e95bc973 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:04:59.650817 28701 raft_consensus.cc:2258] T e1a632c9758343a6890d4bfe6aa383d5 P 6d9ad35e13e3448f922dd7c2e95bc973 [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 11 12:04:59 dist-test-slave-mjjr krb5kdc[27266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449894, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.129@KRBTEST.COM
I20260811 12:04:59.657714 28697 tablet_replica.cc:330] T e1a632c9758343a6890d4bfe6aa383d5 P d65dabfa42a14db4af98d042423ec48f: stopping tablet replica
I20260811 12:04:59.657804 28700 ts_tablet_manager.cc:1918] T e1a632c9758343a6890d4bfe6aa383d5 P 0bb2ac15cbca4eb1b5726e247f355f86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:04:59.658033 28700 log.cc:1196] T e1a632c9758343a6890d4bfe6aa383d5 P 0bb2ac15cbca4eb1b5726e247f355f86: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-0/wal/wals/e1a632c9758343a6890d4bfe6aa383d5
I20260811 12:04:59.658213 28697 raft_consensus.cc:2229] T e1a632c9758343a6890d4bfe6aa383d5 P d65dabfa42a14db4af98d042423ec48f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:04:59.658747 28700 ts_tablet_manager.cc:1939] T e1a632c9758343a6890d4bfe6aa383d5 P 0bb2ac15cbca4eb1b5726e247f355f86: Deleting consensus metadata
I20260811 12:04:59.658939 28697 raft_consensus.cc:2258] T e1a632c9758343a6890d4bfe6aa383d5 P d65dabfa42a14db4af98d042423ec48f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:04:59.660883 27932 catalog_manager.cc:4841] TS 0bb2ac15cbca4eb1b5726e247f355f86 (127.9.221.129:45343): tablet e1a632c9758343a6890d4bfe6aa383d5 (table test-table [id=7ecb8fe4af424e14b529ed6247fafd03]) successfully deleted
I20260811 12:04:59.662092 28701 ts_tablet_manager.cc:1905] T e1a632c9758343a6890d4bfe6aa383d5 P 6d9ad35e13e3448f922dd7c2e95bc973: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:04:59.663184 28697 ts_tablet_manager.cc:1905] T e1a632c9758343a6890d4bfe6aa383d5 P d65dabfa42a14db4af98d042423ec48f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:04:59.669178 28652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:59.631297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42992 (local address 127.9.221.129:45343)
0811 12:04:59.631553 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 12:04:59.631562 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:04:59.640772 (+ 9210us) server_negotiation.cc:407] Connection header received
0811 12:04:59.641393 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:04:59.641400 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:04:59.641513 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:04:59.641597 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:04:59.644124 (+ 2527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:59.645332 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:59.649237 (+ 3905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:59.650035 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:59.650143 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:59.658208 (+ 8065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:04:59.658235 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:04:59.658261 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:04:59.658316 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:04:59.664302 (+ 5986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:04:59.665394 (+ 1092us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:04:59.665404 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:04:59.665408 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:04:59.665690 (+ 282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:04:59.666384 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:04:59.666391 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:04:59.666395 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:04:59.667037 (+ 642us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:04:59.667416 (+ 379us) server_negotiation.cc:1035] Waiting for connection context
0811 12:04:59.668183 (+ 767us) server_negotiation.cc:299] Negotiation successful
0811 12:04:59.668514 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 12:04:59.671296 28225 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:04:59.672964 28701 ts_tablet_manager.cc:1918] T e1a632c9758343a6890d4bfe6aa383d5 P 6d9ad35e13e3448f922dd7c2e95bc973: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:04:59.673192 28701 log.cc:1196] T e1a632c9758343a6890d4bfe6aa383d5 P 6d9ad35e13e3448f922dd7c2e95bc973: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-1/wal/wals/e1a632c9758343a6890d4bfe6aa383d5
I20260811 12:04:59.674121 28701 ts_tablet_manager.cc:1939] T e1a632c9758343a6890d4bfe6aa383d5 P 6d9ad35e13e3448f922dd7c2e95bc973: Deleting consensus metadata
I20260811 12:04:59.677812 27932 catalog_manager.cc:4841] TS 6d9ad35e13e3448f922dd7c2e95bc973 (127.9.221.130:38017): tablet e1a632c9758343a6890d4bfe6aa383d5 (table test-table [id=7ecb8fe4af424e14b529ed6247fafd03]) successfully deleted
I20260811 12:04:59.678823 28697 ts_tablet_manager.cc:1918] T e1a632c9758343a6890d4bfe6aa383d5 P d65dabfa42a14db4af98d042423ec48f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:04:59.679054 28697 log.cc:1196] T e1a632c9758343a6890d4bfe6aa383d5 P d65dabfa42a14db4af98d042423ec48f: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449743084027-10102-0/minicluster-data/ts-2/wal/wals/e1a632c9758343a6890d4bfe6aa383d5
I20260811 12:04:59.679850 28697 ts_tablet_manager.cc:1939] T e1a632c9758343a6890d4bfe6aa383d5 P d65dabfa42a14db4af98d042423ec48f: Deleting consensus metadata
I20260811 12:04:59.682199 27931 catalog_manager.cc:4841] TS d65dabfa42a14db4af98d042423ec48f (127.9.221.131:44667): tablet e1a632c9758343a6890d4bfe6aa383d5 (table test-table [id=7ecb8fe4af424e14b529ed6247fafd03]) successfully deleted
I20260811 12:04:59.769069 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 28053
I20260811 12:04:59.794175 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 28247
Aug 11 12:04:59 dist-test-slave-mjjr krb5kdc[27266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449894, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.130@KRBTEST.COM
I20260811 12:04:59.817086 28656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:59.795051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49748 (local address 127.9.221.130:38017)
0811 12:04:59.795225 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 12:04:59.795229 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:04:59.795290 (+ 61us) server_negotiation.cc:407] Connection header received
0811 12:04:59.795492 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:04:59.795496 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:04:59.795579 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:04:59.795736 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:04:59.797398 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:59.798169 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:59.800104 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:59.800541 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:59.800637 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:59.810335 (+ 9698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:04:59.810349 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:04:59.810365 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:04:59.810423 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:04:59.813884 (+ 3461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:04:59.814722 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:04:59.814738 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:04:59.814740 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:04:59.814903 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:04:59.815292 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:04:59.815297 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:04:59.815302 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:04:59.815721 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:04:59.816085 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0811 12:04:59.816408 (+ 323us) server_negotiation.cc:299] Negotiation successful
0811 12:04:59.816655 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 12:04:59.818045 28422 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:04:59.937002 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 28247
I20260811 12:04:59.968353 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 28438
Aug 11 12:04:59 dist-test-slave-mjjr krb5kdc[27266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449894, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.131@KRBTEST.COM
I20260811 12:04:59.984105 28650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:04:59.969182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52058 (local address 127.9.221.131:44667)
0811 12:04:59.969375 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 12:04:59.969383 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:04:59.969433 (+ 50us) server_negotiation.cc:407] Connection header received
0811 12:04:59.969633 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:04:59.969639 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:04:59.969729 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:04:59.969834 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:04:59.971121 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:59.972274 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:59.973575 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:04:59.974199 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:04:59.974330 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:04:59.978780 (+ 4450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:04:59.978795 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:04:59.978811 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:04:59.978846 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:04:59.981562 (+ 2716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:04:59.982209 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:04:59.982216 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:04:59.982218 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:04:59.982348 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:04:59.982670 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:04:59.982673 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:04:59.982675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:04:59.983047 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:04:59.983263 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0811 12:04:59.983538 (+ 275us) server_negotiation.cc:299] Negotiation successful
0811 12:04:59.983831 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 12:04:59.984766 28619 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:05:00.075008 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 28438
I20260811 12:05:00.099682 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 27907
I20260811 12:05:00.100906 27981 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:05:00.215859 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 27907
I20260811 12:05:00.250703 10102 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 12:05:00.597048 10102 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 12:05:00.597383 10102 mini_ranger.cc:66] Stopping Ranger...
I20260811 12:05:00.952884 10102 mini_ranger.cc:68] Stopped Ranger
2026-08-11 12:05:00.953 UTC [27308] LOG: received smart shutdown request
2026-08-11 12:05:00.957 UTC [27308] LOG: background worker "logical replication launcher" (PID 27320) exited with exit code 1
2026-08-11 12:05:00.957 UTC [27312] LOG: shutting down
2026-08-11 12:05:01.093 UTC [27308] LOG: database system is shut down
2026-08-11T12:05:01Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (52513 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:05:01 dist-test-slave-mjjr krb5kdc[28726](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:05:01 dist-test-slave-mjjr krb5kdc[28726](info): set up 2 sockets
Aug 11 12:05:01 dist-test-slave-mjjr krb5kdc[28726](info): commencing operation
krb5kdc: starting...
W20260811 12:05:02.618112 10102 mini_kdc.cc:121] Time spent starting KDC: real 1.335s user 0.002s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Aug 11 12:05:02 dist-test-slave-mjjr krb5kdc[28726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:05:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:05:02Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.9.221.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.9.221.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.9.221.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/rangeradmin_127.9.221.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.9.221.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/rangeradmin_127.9.221.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.9.221.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.9.221.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.9.221.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/rangerlookup_127.9.221.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.9.221.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/rangerlookup_127.9.221.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.221.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.9.221.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.221.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/HTTP_127.9.221.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.221.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/HTTP_127.9.221.148@KRBTEST.COM.keytab.
I20260811 12:05:02.974359 10102 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskGvgjmQ/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/postgres -l logfile start
2026-08-11 12:05:05.301 UTC [28766] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 12:05:05.301 UTC [28766] LOG: listening on IPv4 address "127.9.221.148", port 33555
2026-08-11 12:05:05.304 UTC [28766] LOG: listening on Unix socket "/tmp/.s.PGSQL.33555"
2026-08-11 12:05:05.326 UTC [28769] LOG: database system was shut down at 2026-08-11 12:05:04 UTC
2026-08-11 12:05:05.329 UTC [28766] LOG: database system is ready to accept connections
I20260811 12:05:06.529685 10102 mini_postgres.cc:96] Postgres bound to 33555
2026-08-11 12:05:06.551 UTC [28779] FATAL: database "slave" does not exist
127.9.221.148:33555 - accepting connections
I20260811 12:05:06.552009 10102 mini_ranger.cc:161] Starting Ranger...
I20260811 12:05:06.580287 10102 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 12:05:07.009583 10102 mini_ranger.cc:87] Created ranger Postgres database
I20260811 12:05:07.009712 10102 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ranger-admin
2026-08-11 12:05:07,298 [I] DB FLAVOR :POSTGRES
2026-08-11 12:05:07,299 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 12:05:07,299 [I] Checking connection..
2026-08-11 12:05:07,299 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.221.148:33555/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 12:05:07,564 [I] Checking connection passed.
2026-08-11 12:05:07,565 [I] --------- Verifying version history table ---------
2026-08-11 12:05:07,565 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.221.148:33555/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 12:05:07,834 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 12:05:07,834 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.221.148:33555/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 12:05:08,113 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 12:05:08,114 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 12:05:08,114 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.221.148:33555/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 12:05:08.357 UTC [28868] WARNING: there is no transaction in progress
2026-08-11 12:05:08,366 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 12:05:08,367 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.221.148:33555/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 12:05:08,614 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 12:05:08,614 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 12:05:08,614 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.221.148:33555/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 12:05:08,868 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.221.148:33555/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 12:05:09,133 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.221.148:33555/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-mjjr.c.gcp-upstream.internal','N') ;"
2026-08-11 12:05:09,362 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.221.148:33555/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 12:05:09,626 [I] Table x_portal_user does not exist in database ranger
2026-08-11 12:05:09,626 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.221.148:33555/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 12:05:09,878 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 12:05:09,879 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.221.148:33555/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 12:05:10,137 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.221.148:33555/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 12:05:10,399 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 12:05:10,399 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.221.148:33555/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 12:05:11.292 UTC [29015] WARNING: there is no transaction in progress
2026-08-11 12:05:11.307 UTC [29015] WARNING: there is no transaction in progress
2026-08-11 12:05:11.322 UTC [29015] WARNING: there is no transaction in progress
2026-08-11 12:05:11.335 UTC [29015] WARNING: there is no transaction in progress
2026-08-11 12:05:11.349 UTC [29015] WARNING: there is no transaction in progress
2026-08-11 12:05:11.367 UTC [29015] WARNING: there is no transaction in progress
2026-08-11 12:05:11.429 UTC [29015] WARNING: there is no transaction in progress
2026-08-11 12:05:11.435 UTC [29015] WARNING: there is no transaction in progress
2026-08-11 12:05:11.447 UTC [29015] WARNING: there is no transaction in progress
2026-08-11 12:05:11.456 UTC [29015] WARNING: there is no transaction in progress
2026-08-11 12:05:11.465 UTC [29015] WARNING: there is no transaction in progress
2026-08-11 12:05:11.478 UTC [29015] WARNING: there is no transaction in progress
2026-08-11 12:05:11.487 UTC [29015] WARNING: there is no transaction in progress
2026-08-11 12:05:11.493 UTC [29015] WARNING: there is no transaction in progress
2026-08-11 12:05:11.499 UTC [29015] WARNING: there is no transaction in progress
2026-08-11 12:05:11.504 UTC [29015] WARNING: there is no transaction in progress
2026-08-11 12:05:12,053 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 12:05:12,054 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.221.148:33555/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 12:05:12,312 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 12:05:12,313 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.221.148:33555/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 12:05:12,550 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 12:05:12,550 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.221.148:33555/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 12:05:12,791 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 12:05:12,792 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.221.148:33555/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 12:05:13,054 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.221.148:33555/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 12:05:13,313 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.221.148:33555/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-mjjr.c.gcp-upstream.internal';"
2026-08-11 12:05:13,565 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 12:05:13,566 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.221.148:33555/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 12:05:13,813 [I] DB_PATCHES have already been applied
I20260811 12:05:13.819851 10102 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ranger-admin:/tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskGvgjmQ/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 12:05:13.819952 10102 mini_ranger.cc:193] Using host: 127.9.221.148
I20260811 12:05:13.838788 10102 mini_ranger.cc:227] Ranger admin URL: http://127.9.221.148:53025
Aug 11, 2026 12:05:14 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 12:05:14 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 12:05:14 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 12:05:14 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 12:05:14 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 12:05:14 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 12:05:14 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 12:05:15.261759 10102 mini_ranger.cc:160] Time spent starting Ranger: real 8.710s user 0.004s sys 0.046s
Aug 11, 2026 12:05:34 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 12:05:34 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 12:05:34 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 12:05:34 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 12:05:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 12:05:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 12:05:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 12:05:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 12:05:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 12:05:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 12:05:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 12:05:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 12:05:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 12:05:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 12:05:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 12:05:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 12:05:34 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 12:05:35 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 12:05:35.861327 10102 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.9.221.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.9.221.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.9.221.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/rangerkms_127.9.221.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.9.221.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/rangerkms_127.9.221.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.221.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.9.221.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.221.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/HTTP_127.9.221.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.221.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/HTTP_127.9.221.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 12:05:36 dist-test-slave-mjjr krb5kdc[28726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449936, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 12:05:36.082919 10102 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 12:05:36.108093 10102 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 12:05:36.859211 10102 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 12:05:36.859349 10102 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ranger-kms
I20260811 12:05:36.859400 10102 mini_ranger_kms.cc:227] Using postgres at 127.9.221.148:33555
2026-08-11 12:05:37,228 [I] DB FLAVOR :POSTGRES
2026-08-11 12:05:37,229 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 12:05:37,229 [I] Checking connection
2026-08-11 12:05:37,229 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.221.148:33555/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 12:05:37,479 [I] connection success
2026-08-11 12:05:37,479 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 12:05:37,479 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 12:05:37,480 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.221.148:33555/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 12:05:37,712 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 12:05:37,712 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 12:05:37,713 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 12:05:37,713 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.221.148:33555/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 12:05:37,977 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 12:05:38.468538 10102 mini_ranger_kms.cc:310] Created kms service
I20260811 12:05:38.664352 10102 mini_ranger_kms.cc:326] Created kudu user
I20260811 12:05:38.709970 10102 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 12:05:38 dist-test-slave-mjjr krb5kdc[28726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449936, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.9.221.148@KRBTEST.COM
I20260811 12:05:39.126003 10102 mini_ranger_kms.cc:384] Added ranger policy
I20260811 12:05:39.126377 10102 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ranger-kms:/tmp/dist-test-taskGvgjmQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskGvgjmQ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 12:05:39.126447 10102 mini_ranger_kms.cc:240] Using host: 127.9.221.148
I20260811 12:05:39.148787 10102 mini_ranger_kms.cc:276] Ranger KMS PID: 29272
I20260811 12:05:39.148924 10102 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.9.221.148:50435
Aug 11, 2026 12:05:40 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 12:05:40 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 12:05:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 12:05:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 12:05:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 12:05:40 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 12:05:41 dist-test-slave-mjjr krb5kdc[28726](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786449941, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.221.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 11 12:05:41 dist-test-slave-mjjr krb5kdc[28726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449941, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.221.148@KRBTEST.COM for HTTP/127.9.221.148@KRBTEST.COM
I20260811 12:05:41.963115 10102 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.880s user 0.059s sys 0.341s
I20260811 12:05:41.963390 10102 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 12:05:41.963593 10102 mini_ranger_kms.cc:401] 127.9.221.148:50435/kms/v1/keys
Aug 11 12:05:42 dist-test-slave-mjjr krb5kdc[28726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449941, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.221.148@KRBTEST.COM for HTTP/127.9.221.148@KRBTEST.COM
Aug 11 12:05:44 dist-test-slave-mjjr krb5kdc[28726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.9.221.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
I20260811 12:05:44.663902 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:46125
--webserver_interface=127.9.221.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:38461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:46125
--ranger_config_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.221.148:50435/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:05:45.077558 29339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:05:45.077888 29339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:05:45.078049 29339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:05:45.087293 29339 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:05:45.087440 29339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:05:45.087528 29339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:05:45.087638 29339 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:05:45.087735 29339 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:05:45.099936 29339 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:38461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/master-0/wal
--ranger_kms_url=127.9.221.148:50435/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:46125
--ranger_config_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:46125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.221.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:05:45.101425 29339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:05:45.102804 29339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:05:45.114892 29348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:05:45.114956 29347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:05:45.114996 29351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:05:45.115758 29339 server_base.cc:1034] running on GCE node
I20260811 12:05:45.116487 29339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:05:45.117841 29339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:05:45.119071 29339 hybrid_clock.cc:648] HybridClock initialized: now 1786449945119039 us; error 52 us; skew 500 ppm
I20260811 12:05:45.119421 29339 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:05:45 dist-test-slave-mjjr krb5kdc[28726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449945, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:05:45.122794 29339 init.cc:377] Logged in from keytab as kudu/127.9.221.190@KRBTEST.COM (short username kudu)
I20260811 12:05:45.124378 29339 webserver.cc:460] Webserver started at http://127.9.221.190:43553/ using document root <none> and password file <none>
I20260811 12:05:45.125025 29339 fs_manager.cc:359] Metadata directory not provided
I20260811 12:05:45.125170 29339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:05:45.125527 29339 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 12:05:45 dist-test-slave-mjjr krb5kdc[28726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449945, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for HTTP/127.9.221.148@KRBTEST.COM
I20260811 12:05:45.189095 29339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "540120c3bbe14057a28352ca4e19d1ce"
format_stamp: "Formatted at 2026-08-11 12:05:45 on dist-test-slave-mjjr"
server_key: "ce2040affbefe39e8c75addc2980e37c"
server_key_iv: "9903700c09d3e4aef80475c90f59afc1"
server_key_version: "kuduclusterkey@0"
I20260811 12:05:45.189992 29339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "540120c3bbe14057a28352ca4e19d1ce"
format_stamp: "Formatted at 2026-08-11 12:05:45 on dist-test-slave-mjjr"
server_key: "ce2040affbefe39e8c75addc2980e37c"
server_key_iv: "9903700c09d3e4aef80475c90f59afc1"
server_key_version: "kuduclusterkey@0"
I20260811 12:05:45.195564 29339 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 12:05:45.211980 29362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:05:45.213531 29339 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 12:05:45.213706 29339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "540120c3bbe14057a28352ca4e19d1ce"
format_stamp: "Formatted at 2026-08-11 12:05:45 on dist-test-slave-mjjr"
server_key: "ce2040affbefe39e8c75addc2980e37c"
server_key_iv: "9903700c09d3e4aef80475c90f59afc1"
server_key_version: "kuduclusterkey@0"
I20260811 12:05:45.213925 29339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:05:45.246017 29339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:05:45.250077 29339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:05:45.250437 29339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:05:45.270658 29339 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.190:46125
I20260811 12:05:45.270669 29436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.190:46125 every 8 connection(s)
I20260811 12:05:45.272082 29339 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:05:45.274304 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 29339
I20260811 12:05:45.274554 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:05:45.275470 10102 external_mini_cluster.cc:1428] Setting key e40a6a85d1c5c9b4a65f87f603aac956
I20260811 12:05:45.276454 29437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:05:45.295037 29437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 540120c3bbe14057a28352ca4e19d1ce: Bootstrap starting.
I20260811 12:05:45.298453 29437 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 540120c3bbe14057a28352ca4e19d1ce: Neither blocks nor log segments found. Creating new log.
I20260811 12:05:45.299809 29437 log.cc:824] T 00000000000000000000000000000000 P 540120c3bbe14057a28352ca4e19d1ce: Log is configured to *not* fsync() on all Append() calls
Aug 11 12:05:45 dist-test-slave-mjjr krb5kdc[28726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:05:45.302901 29437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 540120c3bbe14057a28352ca4e19d1ce: No bootstrap required, opened a new log
I20260811 12:05:45.312250 29437 raft_consensus.cc:348] T 00000000000000000000000000000000 P 540120c3bbe14057a28352ca4e19d1ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "540120c3bbe14057a28352ca4e19d1ce" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 46125 } }
I20260811 12:05:45.312608 29437 raft_consensus.cc:374] T 00000000000000000000000000000000 P 540120c3bbe14057a28352ca4e19d1ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:05:45.312737 29437 raft_consensus.cc:729] T 00000000000000000000000000000000 P 540120c3bbe14057a28352ca4e19d1ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 540120c3bbe14057a28352ca4e19d1ce, State: Initialized, Role: FOLLOWER
I20260811 12:05:45.312891 29440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:45.277785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60124 (local address 127.9.221.190:46125)
0811 12:05:45.288310 (+ 10525us) server_negotiation.cc:206] Beginning negotiation
0811 12:05:45.288330 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 12:05:45.288744 (+ 414us) server_negotiation.cc:407] Connection header received
0811 12:05:45.290866 (+ 2122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:05:45.290897 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:05:45.291442 (+ 545us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:05:45.291940 (+ 498us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:05:45.293574 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:45.295491 (+ 1917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:45.297377 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:45.298440 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:45.298644 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:45.303558 (+ 4914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:05:45.303569 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:05:45.303587 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:05:45.303631 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:05:45.308002 (+ 4371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:45.308879 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:45.308888 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:45.308897 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:45.309086 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:45.309601 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:45.309607 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:45.309614 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:45.310349 (+ 735us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:05:45.310679 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0811 12:05:45.311156 (+ 477us) server_negotiation.cc:299] Negotiation successful
0811 12:05:45.311588 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10249,"thread_start_us":10007,"threads_started":1}
I20260811 12:05:45.313484 29437 consensus_queue.cc:260] T 00000000000000000000000000000000 P 540120c3bbe14057a28352ca4e19d1ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "540120c3bbe14057a28352ca4e19d1ce" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 46125 } }
I20260811 12:05:45.313735 29437 raft_consensus.cc:388] T 00000000000000000000000000000000 P 540120c3bbe14057a28352ca4e19d1ce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:05:45.313902 29437 raft_consensus.cc:482] T 00000000000000000000000000000000 P 540120c3bbe14057a28352ca4e19d1ce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:05:45.314102 29437 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 540120c3bbe14057a28352ca4e19d1ce [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:05:45.316548 29437 raft_consensus.cc:504] T 00000000000000000000000000000000 P 540120c3bbe14057a28352ca4e19d1ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "540120c3bbe14057a28352ca4e19d1ce" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 46125 } }
I20260811 12:05:45.317022 29437 leader_election.cc:302] T 00000000000000000000000000000000 P 540120c3bbe14057a28352ca4e19d1ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 540120c3bbe14057a28352ca4e19d1ce; no voters:
I20260811 12:05:45.317750 29437 leader_election.cc:288] T 00000000000000000000000000000000 P 540120c3bbe14057a28352ca4e19d1ce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:05:45.317848 29446 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 540120c3bbe14057a28352ca4e19d1ce [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:05:45.320065 29446 raft_consensus.cc:686] T 00000000000000000000000000000000 P 540120c3bbe14057a28352ca4e19d1ce [term 1 LEADER]: Becoming Leader. State: Replica: 540120c3bbe14057a28352ca4e19d1ce, State: Running, Role: LEADER
I20260811 12:05:45.320391 29437 sys_catalog.cc:564] T 00000000000000000000000000000000 P 540120c3bbe14057a28352ca4e19d1ce [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:05:45.320683 29446 consensus_queue.cc:237] T 00000000000000000000000000000000 P 540120c3bbe14057a28352ca4e19d1ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "540120c3bbe14057a28352ca4e19d1ce" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 46125 } }
I20260811 12:05:45.323827 29449 sys_catalog.cc:455] T 00000000000000000000000000000000 P 540120c3bbe14057a28352ca4e19d1ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "540120c3bbe14057a28352ca4e19d1ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "540120c3bbe14057a28352ca4e19d1ce" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 46125 } } }
I20260811 12:05:45.324288 29449 sys_catalog.cc:458] T 00000000000000000000000000000000 P 540120c3bbe14057a28352ca4e19d1ce [sys.catalog]: This master's current role is: LEADER
I20260811 12:05:45.324939 29450 sys_catalog.cc:455] T 00000000000000000000000000000000 P 540120c3bbe14057a28352ca4e19d1ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 540120c3bbe14057a28352ca4e19d1ce. Latest consensus state: current_term: 1 leader_uuid: "540120c3bbe14057a28352ca4e19d1ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "540120c3bbe14057a28352ca4e19d1ce" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 46125 } } }
I20260811 12:05:45.325276 29450 sys_catalog.cc:458] T 00000000000000000000000000000000 P 540120c3bbe14057a28352ca4e19d1ce [sys.catalog]: This master's current role is: LEADER
I20260811 12:05:45.336971 29457 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:05:45.342298 29457 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:05:45.354916 29457 catalog_manager.cc:1347] Generated new cluster ID: 43780444e14e45e29b0a31e103ffccd9
I20260811 12:05:45.355034 29457 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:05:45.376037 29457 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:05:45.379125 29457 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:05:45.390023 29437 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 12:05:45.410744 29457 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 540120c3bbe14057a28352ca4e19d1ce: Generated new TSK 0
I20260811 12:05:45.411981 29457 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.221.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab.
Entry for principal kudu/127.9.221.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab.
I20260811 12:05:46.227032 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.221.148:50435/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.129:0
--local_ip_for_outbound_sockets=127.9.221.129
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:46125
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab
--principal=kudu/127.9.221.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:38461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 12:05:46 dist-test-slave-mjjr krb5kdc[28726](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786449946, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 12:05:46.674916 29498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:05:46.675350 29498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:05:46.675575 29498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:05:46.687321 29498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:05:46.687466 29498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:05:46.687691 29498 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.129
I20260811 12:05:46.703732 29498 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:38461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.9.221.148:50435/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab
--principal=kudu/127.9.221.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:46125
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.129
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:05:46.705895 29498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:05:46.707787 29498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:05:46.725621 29513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:05:46.727856 29514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:05:46.735180 29498 server_base.cc:1034] running on GCE node
W20260811 12:05:46.735510 29516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:05:46.736478 29498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:05:46.737955 29498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:05:46.739367 29498 hybrid_clock.cc:648] HybridClock initialized: now 1786449946739237 us; error 111 us; skew 500 ppm
I20260811 12:05:46.739841 29498 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:05:46 dist-test-slave-mjjr krb5kdc[28726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449946, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:05:46.744351 29498 init.cc:377] Logged in from keytab as kudu/127.9.221.129@KRBTEST.COM (short username kudu)
I20260811 12:05:46.746457 29498 webserver.cc:460] Webserver started at http://127.9.221.129:33741/ using document root <none> and password file <none>
I20260811 12:05:46.747469 29498 fs_manager.cc:359] Metadata directory not provided
I20260811 12:05:46.747611 29498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:05:46.748059 29498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 12:05:46 dist-test-slave-mjjr krb5kdc[28726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449946, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for HTTP/127.9.221.148@KRBTEST.COM
I20260811 12:05:46.768502 29498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-0/data/instance:
uuid: "9bb16cd74e3944e08501246c3a2b4628"
format_stamp: "Formatted at 2026-08-11 12:05:46 on dist-test-slave-mjjr"
server_key: "abb0c6e62ef82f9ffddf8f6d3165f429"
server_key_iv: "c2292f3ca812e82e44943dba37c2b234"
server_key_version: "kuduclusterkey@0"
I20260811 12:05:46.769732 29498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance:
uuid: "9bb16cd74e3944e08501246c3a2b4628"
format_stamp: "Formatted at 2026-08-11 12:05:46 on dist-test-slave-mjjr"
server_key: "abb0c6e62ef82f9ffddf8f6d3165f429"
server_key_iv: "c2292f3ca812e82e44943dba37c2b234"
server_key_version: "kuduclusterkey@0"
I20260811 12:05:46.781773 29498 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.007s sys 0.003s
I20260811 12:05:46.816206 29527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:05:46.818619 29498 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.006s sys 0.001s
I20260811 12:05:46.818852 29498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-0/wal
uuid: "9bb16cd74e3944e08501246c3a2b4628"
format_stamp: "Formatted at 2026-08-11 12:05:46 on dist-test-slave-mjjr"
server_key: "abb0c6e62ef82f9ffddf8f6d3165f429"
server_key_iv: "c2292f3ca812e82e44943dba37c2b234"
server_key_version: "kuduclusterkey@0"
I20260811 12:05:46.819157 29498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:05:46.850915 29498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:05:46.856019 29498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:05:46.856389 29498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:05:46.859885 29498 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:05:46.861464 29498 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:05:46.861569 29498 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:05:46.861696 29498 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:05:46.861853 29498 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260811 12:05:46.931936 29498 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.129:40913
I20260811 12:05:46.931978 29692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.129:40913 every 8 connection(s)
I20260811 12:05:46.933858 29498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
I20260811 12:05:46.935891 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 29498
I20260811 12:05:46.936193 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance
I20260811 12:05:46.937358 10102 external_mini_cluster.cc:1428] Setting key 819aeccc04d205b5d7f5a5471b4fde03
Aug 11 12:05:46 dist-test-slave-mjjr krb5kdc[28726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449946, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.221.130@KRBTEST.COM; defaulting to no policy
I20260811 12:05:46.986016 29696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:46.938346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:45295 (local address 127.9.221.190:46125)
0811 12:05:46.960357 (+ 22011us) server_negotiation.cc:206] Beginning negotiation
0811 12:05:46.960367 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:05:46.961714 (+ 1347us) server_negotiation.cc:407] Connection header received
0811 12:05:46.964726 (+ 3012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:05:46.964733 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:05:46.964837 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:05:46.964968 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:05:46.967070 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:46.968377 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:46.971042 (+ 2665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:46.971863 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:46.971984 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:46.977532 (+ 5548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:05:46.977541 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:05:46.977548 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:05:46.977598 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:05:46.981712 (+ 4114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:46.982809 (+ 1097us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:46.982817 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:46.982824 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:46.982990 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:46.983606 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:46.983612 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:46.983618 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:46.984015 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:05:46.984149 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 12:05:46.985247 (+ 1098us) server_negotiation.cc:299] Negotiation successful
0811 12:05:46.985522 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21806,"thread_start_us":21660,"threads_started":1}
I20260811 12:05:46.988075 29695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:46.937641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:46125 (local address 127.9.221.129:45295)
0811 12:05:46.961076 (+ 23435us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:46.961133 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 12:05:46.963929 (+ 2796us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:05:46.965226 (+ 1297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:05:46.965253 (+ 27us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:05:46.965952 (+ 699us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:05:46.966877 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:05:46.966907 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:05:46.968578 (+ 1671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:05:46.968586 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:05:46.970864 (+ 2278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:05:46.970882 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:05:46.972346 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:05:46.972355 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:05:46.972675 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:46.973519 (+ 844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:05:46.973558 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:05:46.977263 (+ 3705us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:05:46.981917 (+ 4654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:05:46.981930 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:05:46.981953 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:05:46.982631 (+ 678us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:05:46.983159 (+ 528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:05:46.983166 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:05:46.983172 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:05:46.983459 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:05:46.984267 (+ 808us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:05:46.984277 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:05:46.984759 (+ 482us) client_negotiation.cc:769] Sending connection context
0811 12:05:46.986313 (+ 1554us) client_negotiation.cc:240] Negotiation successful
0811 12:05:46.986908 (+ 595us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23110,"thread_start_us":22825,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.130@KRBTEST.COM" created.
I20260811 12:05:46.990000 29693 heartbeater.cc:347] Connected to a master server at 127.9.221.190:46125
I20260811 12:05:46.990320 29693 heartbeater.cc:464] Registering TS with master...
I20260811 12:05:46.991204 29693 heartbeater.cc:511] Master 127.9.221.190:46125 requested a full tablet report, sending...
I20260811 12:05:46.993820 29387 ts_manager.cc:194] Registered new tserver with Master: 9bb16cd74e3944e08501246c3a2b4628 (127.9.221.129:40913)
I20260811 12:05:46.996068 29387 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.129@KRBTEST.COM'} at 127.9.221.129:45295
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab.
Entry for principal kudu/127.9.221.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab.
I20260811 12:05:47.022578 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.221.148:50435/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.130:0
--local_ip_for_outbound_sockets=127.9.221.130
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:46125
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab
--principal=kudu/127.9.221.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:38461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 12:05:47 dist-test-slave-mjjr krb5kdc[28726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449946, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for HTTP/127.9.221.148@KRBTEST.COM
Aug 11 12:05:47 dist-test-slave-mjjr krb5kdc[28726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449946, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for HTTP/127.9.221.148@KRBTEST.COM
W20260811 12:05:47.498821 29699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:05:47.499590 29699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:05:47.499821 29699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:05:47.516497 29699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:05:47.516727 29699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:05:47.516985 29699 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.130
I20260811 12:05:47.533845 29699 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:38461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.9.221.148:50435/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab
--principal=kudu/127.9.221.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:46125
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.130
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:05:47.535892 29699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:05:47.537688 29699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:05:47.561846 29712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:05:47.566738 29699 server_base.cc:1034] running on GCE node
W20260811 12:05:47.570890 29715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:05:47.571359 29709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:05:47.572395 29699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:05:47.573814 29699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:05:47.575093 29699 hybrid_clock.cc:648] HybridClock initialized: now 1786449947574962 us; error 118 us; skew 500 ppm
I20260811 12:05:47.575503 29699 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:05:47 dist-test-slave-mjjr krb5kdc[28726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449947, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:05:47.580046 29699 init.cc:377] Logged in from keytab as kudu/127.9.221.130@KRBTEST.COM (short username kudu)
I20260811 12:05:47.603967 29699 webserver.cc:460] Webserver started at http://127.9.221.130:35883/ using document root <none> and password file <none>
I20260811 12:05:47.605185 29699 fs_manager.cc:359] Metadata directory not provided
I20260811 12:05:47.605439 29699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:05:47.605898 29699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 12:05:47 dist-test-slave-mjjr krb5kdc[28726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449947, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.130@KRBTEST.COM for HTTP/127.9.221.148@KRBTEST.COM
I20260811 12:05:47.629082 29699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-1/data/instance:
uuid: "9fbd67d118d54d42b36c2095f1fcca6f"
format_stamp: "Formatted at 2026-08-11 12:05:47 on dist-test-slave-mjjr"
server_key: "a443ea69416fd2f0d5665c731d078736"
server_key_iv: "930611a7462fd9a52d77426e66ab7a20"
server_key_version: "kuduclusterkey@0"
I20260811 12:05:47.630365 29699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance:
uuid: "9fbd67d118d54d42b36c2095f1fcca6f"
format_stamp: "Formatted at 2026-08-11 12:05:47 on dist-test-slave-mjjr"
server_key: "a443ea69416fd2f0d5665c731d078736"
server_key_iv: "930611a7462fd9a52d77426e66ab7a20"
server_key_version: "kuduclusterkey@0"
I20260811 12:05:47.644165 29699 fs_manager.cc:691] Time spent creating directory manager: real 0.013s user 0.008s sys 0.004s
I20260811 12:05:47.680226 29724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:05:47.684355 29699 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.003s sys 0.001s
I20260811 12:05:47.684662 29699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-1/wal
uuid: "9fbd67d118d54d42b36c2095f1fcca6f"
format_stamp: "Formatted at 2026-08-11 12:05:47 on dist-test-slave-mjjr"
server_key: "a443ea69416fd2f0d5665c731d078736"
server_key_iv: "930611a7462fd9a52d77426e66ab7a20"
server_key_version: "kuduclusterkey@0"
I20260811 12:05:47.685045 29699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:05:47.744442 29699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:05:47.750591 29699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:05:47.751081 29699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:05:47.764185 29699 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:05:47.768199 29699 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:05:47.768422 29699 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 12:05:47.768625 29699 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:05:47.768754 29699 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:05:47.849229 29699 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.130:46369
I20260811 12:05:47.849293 29879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.130:46369 every 8 connection(s)
I20260811 12:05:47.851158 29699 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
I20260811 12:05:47.856582 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 29699
I20260811 12:05:47.856773 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance
I20260811 12:05:47.857375 10102 external_mini_cluster.cc:1428] Setting key 8e69c0436b45f8daff4c7659372dad1c
Aug 11 12:05:47 dist-test-slave-mjjr krb5kdc[28726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449947, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.130@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:05:47.880954 29887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:47.854315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:50257 (local address 127.9.221.190:46125)
0811 12:05:47.854889 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0811 12:05:47.854896 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:05:47.859450 (+ 4554us) server_negotiation.cc:407] Connection header received
0811 12:05:47.861708 (+ 2258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:05:47.861713 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:05:47.861818 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:05:47.861912 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:05:47.864225 (+ 2313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:47.865430 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:47.867986 (+ 2556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:47.868610 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:47.868709 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:47.873209 (+ 4500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:05:47.873216 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:05:47.873221 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:05:47.873262 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:05:47.877049 (+ 3787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:47.877988 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:47.877994 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:47.877998 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:47.878159 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:47.878745 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:47.878751 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:47.878754 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:47.879117 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:05:47.879268 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 12:05:47.880316 (+ 1048us) server_negotiation.cc:299] Negotiation successful
0811 12:05:47.880561 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":281,"threads_started":1}
I20260811 12:05:47.883446 29890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:47.857636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:46125 (local address 127.9.221.130:50257)
0811 12:05:47.858980 (+ 1344us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:47.859037 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 12:05:47.861401 (+ 2364us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:05:47.862502 (+ 1101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:05:47.862513 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:05:47.863157 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:05:47.864020 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:05:47.864046 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:05:47.865651 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:05:47.865656 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:05:47.867811 (+ 2155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:05:47.867836 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:05:47.869063 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:05:47.869073 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:05:47.869381 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:47.870176 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:05:47.870211 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:05:47.872954 (+ 2743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:05:47.877242 (+ 4288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:05:47.877251 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:05:47.877268 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:05:47.877847 (+ 579us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:05:47.878332 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:05:47.878336 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:05:47.878340 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:05:47.878611 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:05:47.879384 (+ 773us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:05:47.879392 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:05:47.879848 (+ 456us) client_negotiation.cc:769] Sending connection context
0811 12:05:47.881545 (+ 1697us) client_negotiation.cc:240] Negotiation successful
0811 12:05:47.882076 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1052,"thread_start_us":811,"threads_started":1}
I20260811 12:05:47.887375 29882 heartbeater.cc:347] Connected to a master server at 127.9.221.190:46125
I20260811 12:05:47.887794 29882 heartbeater.cc:464] Registering TS with master...
I20260811 12:05:47.888775 29882 heartbeater.cc:511] Master 127.9.221.190:46125 requested a full tablet report, sending...
I20260811 12:05:47.891264 29387 ts_manager.cc:194] Registered new tserver with Master: 9fbd67d118d54d42b36c2095f1fcca6f (127.9.221.130:46369)
WARNING: no policy specified for kudu/127.9.221.131@KRBTEST.COM; defaulting to no policy
I20260811 12:05:47.892437 29387 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.130@KRBTEST.COM'} at 127.9.221.130:50257
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab.
Entry for principal kudu/127.9.221.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab.
I20260811 12:05:47.925073 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.221.148:50435/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.131:0
--local_ip_for_outbound_sockets=127.9.221.131
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:46125
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab
--principal=kudu/127.9.221.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:38461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:05:47.999917 29693 heartbeater.cc:503] Master 127.9.221.190:46125 was elected leader, sending a full tablet report...
W20260811 12:05:48.280745 29899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:05:48.281113 29899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:05:48.281258 29899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:05:48.293319 29899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:05:48.293473 29899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:05:48.293691 29899 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.131
I20260811 12:05:48.310956 29899 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:38461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.9.221.148:50435/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab
--principal=kudu/127.9.221.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:46125
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.131
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:05:48.312866 29899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:05:48.314538 29899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:05:48.331632 29904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:05:48.332065 29909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:05:48.332453 29906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:05:48.333248 29899 server_base.cc:1034] running on GCE node
I20260811 12:05:48.333958 29899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:05:48.335160 29899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:05:48.336388 29899 hybrid_clock.cc:648] HybridClock initialized: now 1786449948336288 us; error 94 us; skew 500 ppm
I20260811 12:05:48.336716 29899 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:05:48 dist-test-slave-mjjr krb5kdc[28726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449948, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:05:48.339895 29899 init.cc:377] Logged in from keytab as kudu/127.9.221.131@KRBTEST.COM (short username kudu)
I20260811 12:05:48.341500 29899 webserver.cc:460] Webserver started at http://127.9.221.131:33159/ using document root <none> and password file <none>
I20260811 12:05:48.342182 29899 fs_manager.cc:359] Metadata directory not provided
I20260811 12:05:48.342330 29899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:05:48.342653 29899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 12:05:48 dist-test-slave-mjjr krb5kdc[28726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449948, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.131@KRBTEST.COM for HTTP/127.9.221.148@KRBTEST.COM
I20260811 12:05:48.358294 29899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-2/data/instance:
uuid: "5a10bee5143c47a3863ef6edc6ab7f3f"
format_stamp: "Formatted at 2026-08-11 12:05:48 on dist-test-slave-mjjr"
server_key: "27f5c6c70b89202908208e56e14d9d7a"
server_key_iv: "8fcabda097695f6abf164bfc8814e2a5"
server_key_version: "kuduclusterkey@0"
I20260811 12:05:48.359293 29899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance:
uuid: "5a10bee5143c47a3863ef6edc6ab7f3f"
format_stamp: "Formatted at 2026-08-11 12:05:48 on dist-test-slave-mjjr"
server_key: "27f5c6c70b89202908208e56e14d9d7a"
server_key_iv: "8fcabda097695f6abf164bfc8814e2a5"
server_key_version: "kuduclusterkey@0"
I20260811 12:05:48.364879 29899 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.003s
I20260811 12:05:48.381479 29922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:05:48.383149 29899 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 12:05:48.383342 29899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-2/wal
uuid: "5a10bee5143c47a3863ef6edc6ab7f3f"
format_stamp: "Formatted at 2026-08-11 12:05:48 on dist-test-slave-mjjr"
server_key: "27f5c6c70b89202908208e56e14d9d7a"
server_key_iv: "8fcabda097695f6abf164bfc8814e2a5"
server_key_version: "kuduclusterkey@0"
I20260811 12:05:48.383538 29899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:05:48.408901 29899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:05:48.412489 29899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:05:48.412792 29899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:05:48.413769 29899 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:05:48.415181 29899 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:05:48.415256 29899 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:05:48.415432 29899 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:05:48.415501 29899 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:05:48.449664 29899 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.131:43551
I20260811 12:05:48.449709 30088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.131:43551 every 8 connection(s)
I20260811 12:05:48.450940 29899 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
I20260811 12:05:48.453015 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 29899
I20260811 12:05:48.453243 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance
I20260811 12:05:48.453743 10102 external_mini_cluster.cc:1428] Setting key 0ddfeced21a30a03220aa47ccb67b750
Aug 11 12:05:48 dist-test-slave-mjjr krb5kdc[28726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449948, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.131@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:05:48.471676 30092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:48.453990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:37973 (local address 127.9.221.190:46125)
0811 12:05:48.454563 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0811 12:05:48.454571 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:05:48.455475 (+ 904us) server_negotiation.cc:407] Connection header received
0811 12:05:48.457813 (+ 2338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:05:48.457818 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:05:48.457900 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:05:48.458026 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:05:48.459418 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:48.460267 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:48.461800 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:48.462231 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:48.462367 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:48.465752 (+ 3385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:05:48.465757 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:05:48.465760 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:05:48.465791 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:05:48.468479 (+ 2688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:48.469361 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:48.469366 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:48.469370 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:48.469519 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:48.469953 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:48.469958 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:48.469961 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:48.470234 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:05:48.470397 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 12:05:48.471046 (+ 649us) server_negotiation.cc:299] Negotiation successful
0811 12:05:48.471271 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":254,"threads_started":1}
I20260811 12:05:48.472618 30093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:48.454505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:46125 (local address 127.9.221.131:37973)
0811 12:05:48.455333 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:48.455370 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 12:05:48.457565 (+ 2195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:05:48.458272 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:05:48.458280 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:05:48.458694 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:05:48.459252 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:05:48.459269 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:05:48.460416 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:05:48.460421 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:05:48.461646 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:05:48.461665 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:05:48.462465 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:05:48.462475 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:05:48.462732 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:48.463392 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:05:48.463419 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:05:48.465528 (+ 2109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:05:48.468643 (+ 3115us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:05:48.468652 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:05:48.468666 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:05:48.469257 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:05:48.469644 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:05:48.469649 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:05:48.469653 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:05:48.469831 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:05:48.470361 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:05:48.470370 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:05:48.470701 (+ 331us) client_negotiation.cc:769] Sending connection context
0811 12:05:48.470963 (+ 262us) client_negotiation.cc:240] Negotiation successful
0811 12:05:48.471451 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":287,"threads_started":1}
I20260811 12:05:48.473959 30090 heartbeater.cc:347] Connected to a master server at 127.9.221.190:46125
I20260811 12:05:48.474254 30090 heartbeater.cc:464] Registering TS with master...
I20260811 12:05:48.474932 30090 heartbeater.cc:511] Master 127.9.221.190:46125 requested a full tablet report, sending...
I20260811 12:05:48.476346 29387 ts_manager.cc:194] Registered new tserver with Master: 5a10bee5143c47a3863ef6edc6ab7f3f (127.9.221.131:43551)
I20260811 12:05:48.477120 29387 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.131@KRBTEST.COM'} at 127.9.221.131:37973
I20260811 12:05:48.481650 10102 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:05:48.483466 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 29498
Aug 11 12:05:48 dist-test-slave-mjjr krb5kdc[28726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.129@KRBTEST.COM
I20260811 12:05:48.500674 30095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:48.484514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56260 (local address 127.9.221.129:40913)
0811 12:05:48.485148 (+ 634us) server_negotiation.cc:206] Beginning negotiation
0811 12:05:48.485154 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:05:48.485178 (+ 24us) server_negotiation.cc:407] Connection header received
0811 12:05:48.485233 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:05:48.485238 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:05:48.485417 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:05:48.485540 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:05:48.487110 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:48.488297 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:48.489721 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:48.490266 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:48.490360 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:48.495064 (+ 4704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:05:48.495076 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:05:48.495088 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:05:48.495127 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:05:48.497756 (+ 2629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:48.498450 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:48.498458 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:48.498461 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:48.498616 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:48.499034 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:48.499040 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:48.499043 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:48.499384 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:05:48.499741 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0811 12:05:48.500041 (+ 300us) server_negotiation.cc:299] Negotiation successful
0811 12:05:48.500338 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":317,"threads_started":1}
I20260811 12:05:48.501860 29688 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:05:48.585028 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 29498
I20260811 12:05:48.609606 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 29699
Aug 11 12:05:48 dist-test-slave-mjjr krb5kdc[28726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.130@KRBTEST.COM
I20260811 12:05:48.626256 30097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:48.610483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39682 (local address 127.9.221.130:46369)
0811 12:05:48.611060 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0811 12:05:48.611066 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:05:48.611092 (+ 26us) server_negotiation.cc:407] Connection header received
0811 12:05:48.611153 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:05:48.611159 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:05:48.611353 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:05:48.611497 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:05:48.612805 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:48.613808 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:48.615142 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:48.615826 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:48.615928 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:48.620602 (+ 4674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:05:48.620612 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:05:48.620625 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:05:48.620660 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:05:48.623337 (+ 2677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:48.624130 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:48.624136 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:48.624138 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:48.624276 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:48.624684 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:48.624688 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:48.624690 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:48.625022 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:05:48.625328 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0811 12:05:48.625656 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 12:05:48.625921 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":308,"threads_started":1}
I20260811 12:05:48.627594 29870 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:05:48.710574 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 29699
I20260811 12:05:48.737850 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 29899
Aug 11 12:05:48 dist-test-slave-mjjr krb5kdc[28726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.131@KRBTEST.COM
I20260811 12:05:48.755414 30099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:48.738949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37506 (local address 127.9.221.131:43551)
0811 12:05:48.739577 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0811 12:05:48.739584 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:05:48.739614 (+ 30us) server_negotiation.cc:407] Connection header received
0811 12:05:48.739732 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:05:48.739740 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:05:48.739958 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:05:48.740170 (+ 212us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:05:48.741667 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:48.742720 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:48.744128 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:48.744775 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:48.744921 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:48.749598 (+ 4677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:05:48.749609 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:05:48.749624 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:05:48.749658 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:05:48.752467 (+ 2809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:48.753183 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:48.753191 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:48.753193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:48.753333 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:48.753730 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:48.753735 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:48.753738 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:48.754103 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:05:48.754462 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0811 12:05:48.754753 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 12:05:48.755030 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":325,"threads_started":1}
I20260811 12:05:48.756788 30082 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:05:48.843369 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 29899
I20260811 12:05:48.864910 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 29339
I20260811 12:05:48.866279 29429 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:05:48.976399 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 29339
I20260811 12:05:49.014292 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:46125
--webserver_interface=127.9.221.190
--webserver_port=43553
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:38461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:46125
--ranger_config_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.221.148:50435/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.221.148:50435/kms with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:05:49.349141 30104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:05:49.349421 30104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:05:49.349557 30104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:05:49.356882 30104 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:05:49.356984 30104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:05:49.357108 30104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:05:49.357216 30104 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:05:49.357302 30104 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:05:49.367331 30104 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:38461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.221.148:50435/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:46125
--ranger_config_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:46125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.221.190
--webserver_port=43553
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:05:49.368639 30104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:05:49.370072 30104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:05:49.382087 30120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:05:49.382194 30112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:05:49.382218 30116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:05:49.383229 30104 server_base.cc:1034] running on GCE node
I20260811 12:05:49.384061 30104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:05:49.385427 30104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:05:49.386673 30104 hybrid_clock.cc:648] HybridClock initialized: now 1786449949386617 us; error 65 us; skew 500 ppm
I20260811 12:05:49.387041 30104 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:05:49 dist-test-slave-mjjr krb5kdc[28726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449949, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:05:49.393117 30104 init.cc:377] Logged in from keytab as kudu/127.9.221.190@KRBTEST.COM (short username kudu)
I20260811 12:05:49.394660 30104 webserver.cc:460] Webserver started at http://127.9.221.190:43553/ using document root <none> and password file <none>
I20260811 12:05:49.395334 30104 fs_manager.cc:359] Metadata directory not provided
I20260811 12:05:49.395498 30104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 12:05:49 dist-test-slave-mjjr krb5kdc[28726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449949, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for HTTP/127.9.221.148@KRBTEST.COM
I20260811 12:05:49.477939 30104 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 12:05:49.481056 30130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:05:49.482542 30104 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:05:49.482743 30104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "540120c3bbe14057a28352ca4e19d1ce"
format_stamp: "Formatted at 2026-08-11 12:05:45 on dist-test-slave-mjjr"
server_key: "ce2040affbefe39e8c75addc2980e37c"
server_key_iv: "9903700c09d3e4aef80475c90f59afc1"
server_key_version: "kuduclusterkey@0"
I20260811 12:05:49.483757 30104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:05:49.516747 30104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:05:49.520496 30104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:05:49.520805 30104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:05:49.540167 30104 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.190:46125
I20260811 12:05:49.540172 30217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.190:46125 every 8 connection(s)
I20260811 12:05:49.541669 30104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:05:49.548695 30218 sys_catalog.cc:263] Verifying existing consensus state
I20260811 12:05:49.550087 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 30104
I20260811 12:05:49.551112 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.221.148:50435/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.129:40913
--local_ip_for_outbound_sockets=127.9.221.129
--tserver_master_addrs=127.9.221.190:46125
--webserver_port=33741
--webserver_interface=127.9.221.129
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab
--principal=kudu/127.9.221.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:38461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.221.148:50435/kms with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:05:49.552121 30218 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 540120c3bbe14057a28352ca4e19d1ce: Bootstrap starting.
I20260811 12:05:49.571627 30218 log.cc:824] T 00000000000000000000000000000000 P 540120c3bbe14057a28352ca4e19d1ce: Log is configured to *not* fsync() on all Append() calls
I20260811 12:05:49.580892 30218 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 540120c3bbe14057a28352ca4e19d1ce: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 12:05:49.581449 30218 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 540120c3bbe14057a28352ca4e19d1ce: Bootstrap complete.
I20260811 12:05:49.591161 30218 raft_consensus.cc:348] T 00000000000000000000000000000000 P 540120c3bbe14057a28352ca4e19d1ce [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "540120c3bbe14057a28352ca4e19d1ce" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 46125 } }
I20260811 12:05:49.592103 30218 raft_consensus.cc:729] T 00000000000000000000000000000000 P 540120c3bbe14057a28352ca4e19d1ce [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 540120c3bbe14057a28352ca4e19d1ce, State: Initialized, Role: FOLLOWER
I20260811 12:05:49.592746 30218 consensus_queue.cc:260] T 00000000000000000000000000000000 P 540120c3bbe14057a28352ca4e19d1ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "540120c3bbe14057a28352ca4e19d1ce" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 46125 } }
I20260811 12:05:49.592998 30218 raft_consensus.cc:388] T 00000000000000000000000000000000 P 540120c3bbe14057a28352ca4e19d1ce [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:05:49.593130 30218 raft_consensus.cc:482] T 00000000000000000000000000000000 P 540120c3bbe14057a28352ca4e19d1ce [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:05:49.593320 30218 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 540120c3bbe14057a28352ca4e19d1ce [term 1 FOLLOWER]: Advancing to term 2
I20260811 12:05:49.597227 30218 raft_consensus.cc:504] T 00000000000000000000000000000000 P 540120c3bbe14057a28352ca4e19d1ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "540120c3bbe14057a28352ca4e19d1ce" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 46125 } }
I20260811 12:05:49.597704 30218 leader_election.cc:302] T 00000000000000000000000000000000 P 540120c3bbe14057a28352ca4e19d1ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 540120c3bbe14057a28352ca4e19d1ce; no voters:
I20260811 12:05:49.598393 30218 leader_election.cc:288] T 00000000000000000000000000000000 P 540120c3bbe14057a28352ca4e19d1ce [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 12:05:49.598534 30222 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 540120c3bbe14057a28352ca4e19d1ce [term 2 FOLLOWER]: Leader election won for term 2
I20260811 12:05:49.599851 30222 raft_consensus.cc:686] T 00000000000000000000000000000000 P 540120c3bbe14057a28352ca4e19d1ce [term 2 LEADER]: Becoming Leader. State: Replica: 540120c3bbe14057a28352ca4e19d1ce, State: Running, Role: LEADER
I20260811 12:05:49.600461 30222 consensus_queue.cc:237] T 00000000000000000000000000000000 P 540120c3bbe14057a28352ca4e19d1ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "540120c3bbe14057a28352ca4e19d1ce" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 46125 } }
I20260811 12:05:49.600667 30218 sys_catalog.cc:564] T 00000000000000000000000000000000 P 540120c3bbe14057a28352ca4e19d1ce [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:05:49.603029 30225 sys_catalog.cc:455] T 00000000000000000000000000000000 P 540120c3bbe14057a28352ca4e19d1ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 540120c3bbe14057a28352ca4e19d1ce. Latest consensus state: current_term: 2 leader_uuid: "540120c3bbe14057a28352ca4e19d1ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "540120c3bbe14057a28352ca4e19d1ce" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 46125 } } }
I20260811 12:05:49.603431 30225 sys_catalog.cc:458] T 00000000000000000000000000000000 P 540120c3bbe14057a28352ca4e19d1ce [sys.catalog]: This master's current role is: LEADER
I20260811 12:05:49.604131 30223 sys_catalog.cc:455] T 00000000000000000000000000000000 P 540120c3bbe14057a28352ca4e19d1ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "540120c3bbe14057a28352ca4e19d1ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "540120c3bbe14057a28352ca4e19d1ce" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 46125 } } }
I20260811 12:05:49.604429 30223 sys_catalog.cc:458] T 00000000000000000000000000000000 P 540120c3bbe14057a28352ca4e19d1ce [sys.catalog]: This master's current role is: LEADER
I20260811 12:05:49.613108 30230 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:05:49.619127 30230 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:05:49.620085 30218 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 12:05:49.620667 30230 catalog_manager.cc:1259] Loaded cluster ID: 43780444e14e45e29b0a31e103ffccd9
I20260811 12:05:49.620761 30230 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:05:49.624969 30230 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:05:49.626428 30230 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 540120c3bbe14057a28352ca4e19d1ce: Loaded TSK: 0
I20260811 12:05:49.628069 30230 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 12:05:50.025166 30220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:05:50.025588 30220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:05:50.025820 30220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:05:50.037708 30220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:05:50.037927 30220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:05:50.038163 30220 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.129
I20260811 12:05:50.054306 30220 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:38461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.221.148:50435/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab
--principal=kudu/127.9.221.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.129:40913
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.221.129
--webserver_port=33741
--tserver_master_addrs=127.9.221.190:46125
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.129
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:05:50.056396 30220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:05:50.058188 30220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:05:50.105577 30271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:05:50.107167 30272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:05:50.120889 30220 server_base.cc:1034] running on GCE node
W20260811 12:05:50.121881 30274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:05:50.122852 30220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:05:50.124264 30220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:05:50.125589 30220 hybrid_clock.cc:648] HybridClock initialized: now 1786449950125515 us; error 58 us; skew 500 ppm
I20260811 12:05:50.125991 30220 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:05:50 dist-test-slave-mjjr krb5kdc[28726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449950, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:05:50.130508 30220 init.cc:377] Logged in from keytab as kudu/127.9.221.129@KRBTEST.COM (short username kudu)
I20260811 12:05:50.138123 30220 webserver.cc:460] Webserver started at http://127.9.221.129:33741/ using document root <none> and password file <none>
I20260811 12:05:50.139263 30220 fs_manager.cc:359] Metadata directory not provided
I20260811 12:05:50.139493 30220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 12:05:50 dist-test-slave-mjjr krb5kdc[28726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449950, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for HTTP/127.9.221.148@KRBTEST.COM
I20260811 12:05:50.175128 30220 fs_manager.cc:709] Time spent opening directory manager: real 0.011s user 0.001s sys 0.008s
I20260811 12:05:50.188570 30289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:05:50.190486 30220 fs_manager.cc:725] Time spent opening block manager: real 0.014s user 0.009s sys 0.003s
I20260811 12:05:50.190762 30220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-0/wal
uuid: "9bb16cd74e3944e08501246c3a2b4628"
format_stamp: "Formatted at 2026-08-11 12:05:46 on dist-test-slave-mjjr"
server_key: "abb0c6e62ef82f9ffddf8f6d3165f429"
server_key_iv: "c2292f3ca812e82e44943dba37c2b234"
server_key_version: "kuduclusterkey@0"
I20260811 12:05:50.192363 30220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:05:50.213850 30220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:05:50.218727 30220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:05:50.219084 30220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:05:50.227185 30220 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:05:50.230700 30220 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:05:50.230886 30220 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 12:05:50.231088 30220 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:05:50.231244 30220 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:05:50.278177 30220 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.129:40913
I20260811 12:05:50.278225 30448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.129:40913 every 8 connection(s)
I20260811 12:05:50.280131 30220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
I20260811 12:05:50.280761 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 30220
I20260811 12:05:50.281693 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.221.148:50435/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.130:46369
--local_ip_for_outbound_sockets=127.9.221.130
--tserver_master_addrs=127.9.221.190:46125
--webserver_port=35883
--webserver_interface=127.9.221.130
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab
--principal=kudu/127.9.221.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:38461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.221.148:50435/kms with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 12:05:50 dist-test-slave-mjjr krb5kdc[28726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449950, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:05:50.322500 30455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:50.283981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:46125 (local address 127.9.221.129:44589)
0811 12:05:50.296199 (+ 12218us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:50.296261 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 12:05:50.299493 (+ 3232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:05:50.302929 (+ 3436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:05:50.302941 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:05:50.303528 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:05:50.304353 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:05:50.304377 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:05:50.307819 (+ 3442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:05:50.307824 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:05:50.309469 (+ 1645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:05:50.309501 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:05:50.310573 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:05:50.310581 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:05:50.310860 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:50.311615 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:05:50.311681 (+ 66us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:05:50.314251 (+ 2570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:05:50.317906 (+ 3655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:05:50.317915 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:05:50.317929 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:05:50.318452 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:05:50.318921 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:05:50.318926 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:05:50.318930 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:05:50.319119 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:05:50.320016 (+ 897us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:05:50.320025 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:05:50.320337 (+ 312us) client_negotiation.cc:769] Sending connection context
0811 12:05:50.320584 (+ 247us) client_negotiation.cc:240] Negotiation successful
0811 12:05:50.321137 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11826,"thread_start_us":15540,"threads_started":1}
I20260811 12:05:50.322506 30454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:50.283964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:44589 (local address 127.9.221.190:46125)
0811 12:05:50.301145 (+ 17181us) server_negotiation.cc:206] Beginning negotiation
0811 12:05:50.301158 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 12:05:50.301199 (+ 41us) server_negotiation.cc:407] Connection header received
0811 12:05:50.301842 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:05:50.301855 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:05:50.302360 (+ 505us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:05:50.302775 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:05:50.305509 (+ 2734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:50.307589 (+ 2080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:50.309646 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:50.310327 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:50.310538 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:50.314534 (+ 3996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:05:50.314541 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:05:50.314554 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:05:50.314589 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:05:50.317757 (+ 3168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:50.318610 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:50.318617 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:50.318622 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:50.318802 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:50.319247 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:50.319250 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:50.319252 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:50.319858 (+ 606us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:05:50.320071 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 12:05:50.320805 (+ 734us) server_negotiation.cc:299] Negotiation successful
0811 12:05:50.321154 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16949,"thread_start_us":16689,"threads_started":1}
I20260811 12:05:50.326226 30450 heartbeater.cc:347] Connected to a master server at 127.9.221.190:46125
I20260811 12:05:50.326598 30450 heartbeater.cc:464] Registering TS with master...
I20260811 12:05:50.327383 30450 heartbeater.cc:511] Master 127.9.221.190:46125 requested a full tablet report, sending...
W20260811 12:05:50.330816 30450 heartbeater.cc:644] Failed to heartbeat to 127.9.221.190:46125 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 12:05:50.331040 30450 heartbeater.cc:464] Registering TS with master...
I20260811 12:05:50.331312 30450 heartbeater.cc:511] Master 127.9.221.190:46125 requested a full tablet report, sending...
I20260811 12:05:50.332610 30450 heartbeater.cc:464] Registering TS with master...
I20260811 12:05:50.332934 30450 heartbeater.cc:511] Master 127.9.221.190:46125 requested a full tablet report, sending...
W20260811 12:05:50.334009 30450 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 12:05:50.730924 30456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:05:50.731422 30456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:05:50.731647 30456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:05:50.741219 30456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:05:50.741381 30456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:05:50.741595 30456 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.130
I20260811 12:05:50.752729 30456 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:38461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.221.148:50435/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab
--principal=kudu/127.9.221.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.130:46369
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.221.130
--webserver_port=35883
--tserver_master_addrs=127.9.221.190:46125
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.130
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:05:50.754240 30456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:05:50.755838 30456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:05:50.771415 30474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:05:50.771625 30476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:05:50.772171 30473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:05:50.773629 30456 server_base.cc:1034] running on GCE node
I20260811 12:05:50.774387 30456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:05:50.776116 30456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:05:50.777379 30456 hybrid_clock.cc:648] HybridClock initialized: now 1786449950777311 us; error 51 us; skew 500 ppm
I20260811 12:05:50.777755 30456 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:05:50 dist-test-slave-mjjr krb5kdc[28726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449950, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:05:50.782397 30456 init.cc:377] Logged in from keytab as kudu/127.9.221.130@KRBTEST.COM (short username kudu)
I20260811 12:05:50.784996 30456 webserver.cc:460] Webserver started at http://127.9.221.130:35883/ using document root <none> and password file <none>
I20260811 12:05:50.785713 30456 fs_manager.cc:359] Metadata directory not provided
I20260811 12:05:50.785887 30456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 12:05:50 dist-test-slave-mjjr krb5kdc[28726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449950, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.130@KRBTEST.COM for HTTP/127.9.221.148@KRBTEST.COM
I20260811 12:05:50.809628 30456 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 12:05:50.832386 30486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:05:50.834543 30456 fs_manager.cc:725] Time spent opening block manager: real 0.023s user 0.007s sys 0.012s
I20260811 12:05:50.834836 30456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-1/wal
uuid: "9fbd67d118d54d42b36c2095f1fcca6f"
format_stamp: "Formatted at 2026-08-11 12:05:47 on dist-test-slave-mjjr"
server_key: "a443ea69416fd2f0d5665c731d078736"
server_key_iv: "930611a7462fd9a52d77426e66ab7a20"
server_key_version: "kuduclusterkey@0"
I20260811 12:05:50.836472 30456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:05:50.885465 30456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:05:50.890612 30456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:05:50.890954 30456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:05:50.892237 30456 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:05:50.893909 30456 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:05:50.894040 30456 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:05:50.894184 30456 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:05:50.894299 30456 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 12:05:50 dist-test-slave-mjjr krb5kdc[28726](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786449950, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:05:50.946055 30456 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.130:46369
I20260811 12:05:50.946120 30659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.130:46369 every 8 connection(s)
I20260811 12:05:50.947971 30456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
I20260811 12:05:50.949539 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 30456
I20260811 12:05:50.950526 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.221.148:50435/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.131:43551
--local_ip_for_outbound_sockets=127.9.221.131
--tserver_master_addrs=127.9.221.190:46125
--webserver_port=33159
--webserver_interface=127.9.221.131
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab
--principal=kudu/127.9.221.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:38461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.221.148:50435/kms with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 12:05:50 dist-test-slave-mjjr krb5kdc[28726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449950, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.130@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:05:50.997188 30663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:50.951723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:36425 (local address 127.9.221.190:46125)
0811 12:05:50.967222 (+ 15499us) server_negotiation.cc:206] Beginning negotiation
0811 12:05:50.967264 (+ 42us) server_negotiation.cc:399] Waiting for connection header
0811 12:05:50.972968 (+ 5704us) server_negotiation.cc:407] Connection header received
0811 12:05:50.975675 (+ 2707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:05:50.975684 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:05:50.975781 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:05:50.975877 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:05:50.978263 (+ 2386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:50.979549 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:50.982163 (+ 2614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:50.982914 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:50.983031 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:50.988768 (+ 5737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:05:50.988777 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:05:50.988783 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:05:50.988846 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:05:50.992965 (+ 4119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:50.993995 (+ 1030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:50.994000 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:50.994004 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:50.994165 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:50.994759 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:50.994767 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:50.994772 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:50.995199 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:05:50.995366 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 12:05:50.996488 (+ 1122us) server_negotiation.cc:299] Negotiation successful
0811 12:05:50.996759 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15288,"thread_start_us":19922,"threads_started":1}
I20260811 12:05:50.999576 30664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:50.954217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:46125 (local address 127.9.221.130:36425)
0811 12:05:50.972445 (+ 18228us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:50.972505 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 12:05:50.975314 (+ 2809us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:05:50.976519 (+ 1205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:05:50.976534 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:05:50.977206 (+ 672us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:05:50.978045 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:05:50.978072 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:05:50.979796 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:05:50.979804 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:05:50.981984 (+ 2180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:05:50.982011 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:05:50.983458 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:05:50.983468 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:05:50.983843 (+ 375us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:50.984650 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:05:50.984690 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:05:50.988489 (+ 3799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:05:50.993143 (+ 4654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:05:50.993153 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:05:50.993172 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:05:50.993837 (+ 665us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:05:50.994347 (+ 510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:05:50.994355 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:05:50.994362 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:05:50.994613 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:05:50.995483 (+ 870us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:05:50.995493 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:05:50.995973 (+ 480us) client_negotiation.cc:769] Sending connection context
0811 12:05:50.997702 (+ 1729us) client_negotiation.cc:240] Negotiation successful
0811 12:05:50.998233 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17856,"thread_start_us":17575,"threads_started":1}
I20260811 12:05:51.001452 30660 heartbeater.cc:347] Connected to a master server at 127.9.221.190:46125
I20260811 12:05:51.001757 30660 heartbeater.cc:464] Registering TS with master...
I20260811 12:05:51.002560 30660 heartbeater.cc:511] Master 127.9.221.190:46125 requested a full tablet report, sending...
I20260811 12:05:51.004561 30158 ts_manager.cc:194] Registered new tserver with Master: 9fbd67d118d54d42b36c2095f1fcca6f (127.9.221.130:46369)
I20260811 12:05:51.006909 30158 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.130@KRBTEST.COM'} at 127.9.221.130:36425
I20260811 12:05:51.334179 30450 heartbeater.cc:464] Registering TS with master...
I20260811 12:05:51.334728 30450 heartbeater.cc:511] Master 127.9.221.190:46125 requested a full tablet report, sending...
I20260811 12:05:51.336200 30158 ts_manager.cc:194] Registered new tserver with Master: 9bb16cd74e3944e08501246c3a2b4628 (127.9.221.129:40913)
I20260811 12:05:51.337498 30158 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.129@KRBTEST.COM'} at 127.9.221.129:44589
W20260811 12:05:51.379076 30665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:05:51.379405 30665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:05:51.379549 30665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:05:51.388358 30665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:05:51.388531 30665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:05:51.388724 30665 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.131
I20260811 12:05:51.403189 30665 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:38461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.221.148:50435/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab
--principal=kudu/127.9.221.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.131:43551
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.221.131
--webserver_port=33159
--tserver_master_addrs=127.9.221.190:46125
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.131
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:05:51.404947 30665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:05:51.406703 30665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
Aug 11 12:05:51 dist-test-slave-mjjr krb5kdc[28726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449950, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for HTTP/127.9.221.148@KRBTEST.COM
W20260811 12:05:51.426827 30672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:05:51.434600 30673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:05:51.435140 30677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:05:51.447458 30665 server_base.cc:1034] running on GCE node
I20260811 12:05:51.448467 30665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:05:51.452788 30665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:05:51.454061 30665 hybrid_clock.cc:648] HybridClock initialized: now 1786449951453970 us; error 73 us; skew 500 ppm
I20260811 12:05:51.454454 30665 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:05:51 dist-test-slave-mjjr krb5kdc[28726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449951, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:05:51.459450 30665 init.cc:377] Logged in from keytab as kudu/127.9.221.131@KRBTEST.COM (short username kudu)
I20260811 12:05:51.492364 30665 webserver.cc:460] Webserver started at http://127.9.221.131:33159/ using document root <none> and password file <none>
I20260811 12:05:51.493175 30665 fs_manager.cc:359] Metadata directory not provided
I20260811 12:05:51.493345 30665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 12:05:51 dist-test-slave-mjjr krb5kdc[28726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449951, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.131@KRBTEST.COM for HTTP/127.9.221.148@KRBTEST.COM
I20260811 12:05:51.524044 30665 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.002s sys 0.005s
I20260811 12:05:51.527557 30688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:05:51.529430 30665 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 12:05:51.529606 30665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-2/wal
uuid: "5a10bee5143c47a3863ef6edc6ab7f3f"
format_stamp: "Formatted at 2026-08-11 12:05:48 on dist-test-slave-mjjr"
server_key: "27f5c6c70b89202908208e56e14d9d7a"
server_key_iv: "8fcabda097695f6abf164bfc8814e2a5"
server_key_version: "kuduclusterkey@0"
I20260811 12:05:51.530984 30665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:05:51.552174 30665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:05:51.557318 30665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:05:51.557626 30665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:05:51.558828 30665 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:05:51.560582 30665 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:05:51.560691 30665 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:05:51.560793 30665 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:05:51.560873 30665 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:05:51.600980 30665 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.131:43551
I20260811 12:05:51.601058 30858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.131:43551 every 8 connection(s)
I20260811 12:05:51.602864 30665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
I20260811 12:05:51.607443 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 30665
Aug 11 12:05:51 dist-test-slave-mjjr krb5kdc[28726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449951, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.131@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:05:51.639400 30862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:51.607818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:59779 (local address 127.9.221.190:46125)
0811 12:05:51.620051 (+ 12233us) server_negotiation.cc:206] Beginning negotiation
0811 12:05:51.620057 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:05:51.620081 (+ 24us) server_negotiation.cc:407] Connection header received
0811 12:05:51.620145 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:05:51.620150 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:05:51.620255 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:05:51.620370 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:05:51.622361 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:51.623413 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:51.625720 (+ 2307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:51.626356 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:51.626456 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:51.631570 (+ 5114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:05:51.631579 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:05:51.631584 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:05:51.631629 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:05:51.635017 (+ 3388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:51.636598 (+ 1581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:51.636603 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:51.636607 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:51.636753 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:51.637280 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:51.637286 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:51.637289 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:51.637620 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:05:51.637778 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 12:05:51.638693 (+ 915us) server_negotiation.cc:299] Negotiation successful
0811 12:05:51.638919 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12113,"thread_start_us":12013,"threads_started":1}
I20260811 12:05:51.642055 30861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:51.606510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:46125 (local address 127.9.221.131:59779)
0811 12:05:51.612411 (+ 5901us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:51.612460 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 12:05:51.619824 (+ 7364us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:05:51.620780 (+ 956us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:05:51.620792 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:05:51.621380 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:05:51.622126 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:05:51.622148 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:05:51.623600 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:05:51.623605 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:05:51.625549 (+ 1944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:05:51.625565 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:05:51.626829 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:05:51.626835 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:05:51.627083 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:51.627888 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:05:51.627915 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:05:51.631318 (+ 3403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:05:51.635185 (+ 3867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:05:51.635197 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:05:51.635213 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:05:51.636458 (+ 1245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:05:51.636901 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:05:51.636906 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:05:51.636913 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:05:51.637136 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:05:51.637878 (+ 742us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:05:51.637886 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:05:51.638276 (+ 390us) client_negotiation.cc:769] Sending connection context
0811 12:05:51.640097 (+ 1821us) client_negotiation.cc:240] Negotiation successful
0811 12:05:51.640670 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5583,"thread_start_us":5334,"threads_started":1}
I20260811 12:05:51.643975 30859 heartbeater.cc:347] Connected to a master server at 127.9.221.190:46125
I20260811 12:05:51.644299 30859 heartbeater.cc:464] Registering TS with master...
I20260811 12:05:51.645154 30859 heartbeater.cc:511] Master 127.9.221.190:46125 requested a full tablet report, sending...
I20260811 12:05:51.646889 30158 ts_manager.cc:194] Registered new tserver with Master: 5a10bee5143c47a3863ef6edc6ab7f3f (127.9.221.131:43551)
I20260811 12:05:51.647153 30864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:51.620025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60128 (local address 127.9.221.190:46125)
0811 12:05:51.626953 (+ 6928us) server_negotiation.cc:206] Beginning negotiation
0811 12:05:51.626958 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:05:51.626978 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:05:51.627024 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:05:51.627028 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:05:51.627104 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:05:51.627225 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:05:51.629466 (+ 2241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:51.630561 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:51.632523 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:51.635598 (+ 3075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:51.635769 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:51.639200 (+ 3431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:05:51.639207 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:05:51.639212 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:05:51.639257 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:05:51.643044 (+ 3787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:51.644133 (+ 1089us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:51.644138 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:51.644141 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:51.644293 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:51.644764 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:51.644769 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:51.644772 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:51.645091 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:05:51.646427 (+ 1336us) server_negotiation.cc:1035] Waiting for connection context
0811 12:05:51.646568 (+ 141us) server_negotiation.cc:299] Negotiation successful
0811 12:05:51.646766 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6787,"thread_start_us":7727,"threads_started":1}
I20260811 12:05:51.648056 30158 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.131@KRBTEST.COM'} at 127.9.221.131:59779
I20260811 12:05:51.650427 10102 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:05:51.670984 30864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:51.656887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60138 (local address 127.9.221.190:46125)
0811 12:05:51.657064 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 12:05:51.657072 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:05:51.657558 (+ 486us) server_negotiation.cc:407] Connection header received
0811 12:05:51.657825 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:05:51.657831 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:05:51.657917 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:05:51.657987 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:05:51.660425 (+ 2438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:51.661123 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:51.662501 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:51.662932 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:51.663072 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:51.665718 (+ 2646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:05:51.665724 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:05:51.665728 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:05:51.665770 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:05:51.668401 (+ 2631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:51.669027 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:51.669034 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:51.669038 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:51.669165 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:51.669580 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:51.669587 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:51.669590 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:51.669845 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:05:51.670097 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0811 12:05:51.670414 (+ 317us) server_negotiation.cc:299] Negotiation successful
0811 12:05:51.670642 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 12:05:51.676031 30158 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60138:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:05:51.683763 30158 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:05:51.732622 30881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:51.714108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.130:46369 (local address 127.0.0.1:39690)
0811 12:05:51.721435 (+ 7327us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:51.721452 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:05:51.721568 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:05:51.722564 (+ 996us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:05:51.722574 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:05:51.722606 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:05:51.723095 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:05:51.723112 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:05:51.725665 (+ 2553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:05:51.725674 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:05:51.728698 (+ 3024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:05:51.728719 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:05:51.731344 (+ 2625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:05:51.731353 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:05:51.731612 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:51.731696 (+ 84us) client_negotiation.cc:769] Sending connection context
0811 12:05:51.731839 (+ 143us) client_negotiation.cc:240] Negotiation successful
0811 12:05:51.732071 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7219,"thread_start_us":7664,"threads_started":1}
I20260811 12:05:51.735740 30880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:51.712604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39690 (local address 127.9.221.130:46369)
0811 12:05:51.718314 (+ 5710us) server_negotiation.cc:206] Beginning negotiation
0811 12:05:51.718326 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:05:51.721696 (+ 3370us) server_negotiation.cc:407] Connection header received
0811 12:05:51.721807 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:05:51.721814 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:05:51.722016 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:05:51.722211 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:05:51.723345 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:51.725468 (+ 2123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:51.728945 (+ 3477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:51.730768 (+ 1823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:51.730922 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:51.731157 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0811 12:05:51.735208 (+ 4051us) server_negotiation.cc:299] Negotiation successful
0811 12:05:51.735407 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5517,"thread_start_us":5398,"threads_started":1}
I20260811 12:05:51.746160 30881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:51.728366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.131:43551 (local address 127.0.0.1:37514)
0811 12:05:51.733176 (+ 4810us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:51.733200 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 12:05:51.733326 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:05:51.734814 (+ 1488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:05:51.734820 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:05:51.734844 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:05:51.735015 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:05:51.735024 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:05:51.738304 (+ 3280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:05:51.738308 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:05:51.740063 (+ 1755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:05:51.740095 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:05:51.744188 (+ 4093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:05:51.744194 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:05:51.744375 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:51.744412 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 12:05:51.744501 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 12:05:51.745861 (+ 1360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4683,"thread_start_us":11663,"threads_started":1}
I20260811 12:05:51.745930 30563 tablet_service.cc:1467] Processing CreateTablet for tablet e4f329c27da74a15b8e763631ea3ec86 (DEFAULT_TABLE table=test-table [id=69debcf5ffec41adac49efc311bde25e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:05:51.747274 30883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:51.722535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37514 (local address 127.9.221.131:43551)
0811 12:05:51.733906 (+ 11371us) server_negotiation.cc:206] Beginning negotiation
0811 12:05:51.733914 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:05:51.733945 (+ 31us) server_negotiation.cc:407] Connection header received
0811 12:05:51.734054 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:05:51.734063 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:05:51.734330 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:05:51.734517 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:05:51.736230 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:51.738135 (+ 1905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:51.740275 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:51.743713 (+ 3438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:51.743844 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:51.744092 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0811 12:05:51.746730 (+ 2638us) server_negotiation.cc:299] Negotiation successful
0811 12:05:51.746911 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11198,"thread_start_us":11049,"threads_started":1}
I20260811 12:05:51.747781 30563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4f329c27da74a15b8e763631ea3ec86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:05:51.751024 30757 tablet_service.cc:1467] Processing CreateTablet for tablet e4f329c27da74a15b8e763631ea3ec86 (DEFAULT_TABLE table=test-table [id=69debcf5ffec41adac49efc311bde25e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:05:51.752534 30757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4f329c27da74a15b8e763631ea3ec86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:05:51.762876 30878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:51.703411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:40913 (local address 127.0.0.1:56268)
0811 12:05:51.711903 (+ 8492us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:51.711941 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 12:05:51.712116 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:05:51.733904 (+ 21788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:05:51.733909 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:05:51.733937 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:05:51.734294 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:05:51.734306 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:05:51.740716 (+ 6410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:05:51.740724 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:05:51.742644 (+ 1920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:05:51.742658 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:05:51.761730 (+ 19072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:05:51.761740 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:05:51.762012 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:51.762072 (+ 60us) client_negotiation.cc:769] Sending connection context
0811 12:05:51.762156 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 12:05:51.762386 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8202,"thread_start_us":8064,"threads_started":1}
I20260811 12:05:51.765213 30879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:51.703978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56268 (local address 127.9.221.129:40913)
0811 12:05:51.721210 (+ 17232us) server_negotiation.cc:206] Beginning negotiation
0811 12:05:51.721220 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:05:51.721253 (+ 33us) server_negotiation.cc:407] Connection header received
0811 12:05:51.721371 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:05:51.721378 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:05:51.721637 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:05:51.734527 (+ 12890us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:05:51.734834 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:51.736566 (+ 1732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:51.758577 (+ 22011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:51.761356 (+ 2779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:51.763551 (+ 2195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:51.763903 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0811 12:05:51.764017 (+ 114us) server_negotiation.cc:299] Negotiation successful
0811 12:05:51.764482 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17063,"thread_start_us":16924,"threads_started":1}
I20260811 12:05:51.767412 30887 tablet_bootstrap.cc:492] T e4f329c27da74a15b8e763631ea3ec86 P 9fbd67d118d54d42b36c2095f1fcca6f: Bootstrap starting.
I20260811 12:05:51.769773 30889 tablet_bootstrap.cc:492] T e4f329c27da74a15b8e763631ea3ec86 P 5a10bee5143c47a3863ef6edc6ab7f3f: Bootstrap starting.
I20260811 12:05:51.771103 30350 tablet_service.cc:1467] Processing CreateTablet for tablet e4f329c27da74a15b8e763631ea3ec86 (DEFAULT_TABLE table=test-table [id=69debcf5ffec41adac49efc311bde25e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:05:51.772297 30350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4f329c27da74a15b8e763631ea3ec86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:05:51.772372 30887 tablet_bootstrap.cc:654] T e4f329c27da74a15b8e763631ea3ec86 P 9fbd67d118d54d42b36c2095f1fcca6f: Neither blocks nor log segments found. Creating new log.
I20260811 12:05:51.773269 30889 tablet_bootstrap.cc:654] T e4f329c27da74a15b8e763631ea3ec86 P 5a10bee5143c47a3863ef6edc6ab7f3f: Neither blocks nor log segments found. Creating new log.
I20260811 12:05:51.773690 30887 log.cc:824] T e4f329c27da74a15b8e763631ea3ec86 P 9fbd67d118d54d42b36c2095f1fcca6f: Log is configured to *not* fsync() on all Append() calls
I20260811 12:05:51.777688 30887 tablet_bootstrap.cc:492] T e4f329c27da74a15b8e763631ea3ec86 P 9fbd67d118d54d42b36c2095f1fcca6f: No bootstrap required, opened a new log
Aug 11 12:05:51 dist-test-slave-mjjr krb5kdc[28726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449950, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for HTTP/127.9.221.148@KRBTEST.COM
I20260811 12:05:51.778239 30887 ts_tablet_manager.cc:1397] T e4f329c27da74a15b8e763631ea3ec86 P 9fbd67d118d54d42b36c2095f1fcca6f: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260811 12:05:51.774996 30889 log.cc:824] T e4f329c27da74a15b8e763631ea3ec86 P 5a10bee5143c47a3863ef6edc6ab7f3f: Log is configured to *not* fsync() on all Append() calls
I20260811 12:05:51.790149 30889 tablet_bootstrap.cc:492] T e4f329c27da74a15b8e763631ea3ec86 P 5a10bee5143c47a3863ef6edc6ab7f3f: No bootstrap required, opened a new log
I20260811 12:05:51.790540 30889 ts_tablet_manager.cc:1397] T e4f329c27da74a15b8e763631ea3ec86 P 5a10bee5143c47a3863ef6edc6ab7f3f: Time spent bootstrapping tablet: real 0.021s user 0.008s sys 0.000s
I20260811 12:05:51.792281 30887 raft_consensus.cc:348] T e4f329c27da74a15b8e763631ea3ec86 P 9fbd67d118d54d42b36c2095f1fcca6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bb16cd74e3944e08501246c3a2b4628" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 40913 } } peers { permanent_uuid: "9fbd67d118d54d42b36c2095f1fcca6f" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46369 } } peers { permanent_uuid: "5a10bee5143c47a3863ef6edc6ab7f3f" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43551 } }
I20260811 12:05:51.792815 30887 raft_consensus.cc:374] T e4f329c27da74a15b8e763631ea3ec86 P 9fbd67d118d54d42b36c2095f1fcca6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:05:51.792969 30887 raft_consensus.cc:729] T e4f329c27da74a15b8e763631ea3ec86 P 9fbd67d118d54d42b36c2095f1fcca6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fbd67d118d54d42b36c2095f1fcca6f, State: Initialized, Role: FOLLOWER
I20260811 12:05:51.793769 30887 consensus_queue.cc:260] T e4f329c27da74a15b8e763631ea3ec86 P 9fbd67d118d54d42b36c2095f1fcca6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bb16cd74e3944e08501246c3a2b4628" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 40913 } } peers { permanent_uuid: "9fbd67d118d54d42b36c2095f1fcca6f" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46369 } } peers { permanent_uuid: "5a10bee5143c47a3863ef6edc6ab7f3f" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43551 } }
I20260811 12:05:51.806648 30894 tablet_bootstrap.cc:492] T e4f329c27da74a15b8e763631ea3ec86 P 9bb16cd74e3944e08501246c3a2b4628: Bootstrap starting.
I20260811 12:05:51.814199 30889 raft_consensus.cc:348] T e4f329c27da74a15b8e763631ea3ec86 P 5a10bee5143c47a3863ef6edc6ab7f3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bb16cd74e3944e08501246c3a2b4628" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 40913 } } peers { permanent_uuid: "9fbd67d118d54d42b36c2095f1fcca6f" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46369 } } peers { permanent_uuid: "5a10bee5143c47a3863ef6edc6ab7f3f" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43551 } }
I20260811 12:05:51.815764 30660 heartbeater.cc:503] Master 127.9.221.190:46125 was elected leader, sending a full tablet report...
I20260811 12:05:51.817953 30887 ts_tablet_manager.cc:1428] T e4f329c27da74a15b8e763631ea3ec86 P 9fbd67d118d54d42b36c2095f1fcca6f: Time spent starting tablet: real 0.040s user 0.024s sys 0.008s
I20260811 12:05:51.822113 30894 tablet_bootstrap.cc:654] T e4f329c27da74a15b8e763631ea3ec86 P 9bb16cd74e3944e08501246c3a2b4628: Neither blocks nor log segments found. Creating new log.
I20260811 12:05:51.822428 30889 raft_consensus.cc:374] T e4f329c27da74a15b8e763631ea3ec86 P 5a10bee5143c47a3863ef6edc6ab7f3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:05:51.822643 30889 raft_consensus.cc:729] T e4f329c27da74a15b8e763631ea3ec86 P 5a10bee5143c47a3863ef6edc6ab7f3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a10bee5143c47a3863ef6edc6ab7f3f, State: Initialized, Role: FOLLOWER
I20260811 12:05:51.823521 30889 consensus_queue.cc:260] T e4f329c27da74a15b8e763631ea3ec86 P 5a10bee5143c47a3863ef6edc6ab7f3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bb16cd74e3944e08501246c3a2b4628" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 40913 } } peers { permanent_uuid: "9fbd67d118d54d42b36c2095f1fcca6f" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46369 } } peers { permanent_uuid: "5a10bee5143c47a3863ef6edc6ab7f3f" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43551 } }
I20260811 12:05:51.825555 30894 log.cc:824] T e4f329c27da74a15b8e763631ea3ec86 P 9bb16cd74e3944e08501246c3a2b4628: Log is configured to *not* fsync() on all Append() calls
I20260811 12:05:51.832335 30889 ts_tablet_manager.cc:1428] T e4f329c27da74a15b8e763631ea3ec86 P 5a10bee5143c47a3863ef6edc6ab7f3f: Time spent starting tablet: real 0.042s user 0.019s sys 0.004s
I20260811 12:05:51.832913 30859 heartbeater.cc:503] Master 127.9.221.190:46125 was elected leader, sending a full tablet report...
I20260811 12:05:51.834245 30894 tablet_bootstrap.cc:492] T e4f329c27da74a15b8e763631ea3ec86 P 9bb16cd74e3944e08501246c3a2b4628: No bootstrap required, opened a new log
I20260811 12:05:51.834611 30894 ts_tablet_manager.cc:1397] T e4f329c27da74a15b8e763631ea3ec86 P 9bb16cd74e3944e08501246c3a2b4628: Time spent bootstrapping tablet: real 0.028s user 0.009s sys 0.005s
I20260811 12:05:51.847343 30894 raft_consensus.cc:348] T e4f329c27da74a15b8e763631ea3ec86 P 9bb16cd74e3944e08501246c3a2b4628 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bb16cd74e3944e08501246c3a2b4628" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 40913 } } peers { permanent_uuid: "9fbd67d118d54d42b36c2095f1fcca6f" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46369 } } peers { permanent_uuid: "5a10bee5143c47a3863ef6edc6ab7f3f" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43551 } }
I20260811 12:05:51.847944 30894 raft_consensus.cc:374] T e4f329c27da74a15b8e763631ea3ec86 P 9bb16cd74e3944e08501246c3a2b4628 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:05:51.848093 30894 raft_consensus.cc:729] T e4f329c27da74a15b8e763631ea3ec86 P 9bb16cd74e3944e08501246c3a2b4628 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bb16cd74e3944e08501246c3a2b4628, State: Initialized, Role: FOLLOWER
I20260811 12:05:51.848788 30894 consensus_queue.cc:260] T e4f329c27da74a15b8e763631ea3ec86 P 9bb16cd74e3944e08501246c3a2b4628 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bb16cd74e3944e08501246c3a2b4628" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 40913 } } peers { permanent_uuid: "9fbd67d118d54d42b36c2095f1fcca6f" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46369 } } peers { permanent_uuid: "5a10bee5143c47a3863ef6edc6ab7f3f" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43551 } }
I20260811 12:05:51.852860 30450 heartbeater.cc:503] Master 127.9.221.190:46125 was elected leader, sending a full tablet report...
I20260811 12:05:51.853680 30894 ts_tablet_manager.cc:1428] T e4f329c27da74a15b8e763631ea3ec86 P 9bb16cd74e3944e08501246c3a2b4628: Time spent starting tablet: real 0.019s user 0.016s sys 0.000s
W20260811 12:05:51.867403 30860 tablet.cc:2369] T e4f329c27da74a15b8e763631ea3ec86 P 5a10bee5143c47a3863ef6edc6ab7f3f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:05:51.978111 30661 tablet.cc:2369] T e4f329c27da74a15b8e763631ea3ec86 P 9fbd67d118d54d42b36c2095f1fcca6f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:05:52.033203 30452 tablet.cc:2369] T e4f329c27da74a15b8e763631ea3ec86 P 9bb16cd74e3944e08501246c3a2b4628: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:05:52.054546 30897 raft_consensus.cc:482] T e4f329c27da74a15b8e763631ea3ec86 P 5a10bee5143c47a3863ef6edc6ab7f3f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:05:52.054900 30897 raft_consensus.cc:504] T e4f329c27da74a15b8e763631ea3ec86 P 5a10bee5143c47a3863ef6edc6ab7f3f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bb16cd74e3944e08501246c3a2b4628" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 40913 } } peers { permanent_uuid: "9fbd67d118d54d42b36c2095f1fcca6f" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46369 } } peers { permanent_uuid: "5a10bee5143c47a3863ef6edc6ab7f3f" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43551 } }
I20260811 12:05:52.056852 30897 leader_election.cc:288] T e4f329c27da74a15b8e763631ea3ec86 P 5a10bee5143c47a3863ef6edc6ab7f3f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9bb16cd74e3944e08501246c3a2b4628 (127.9.221.129:40913), 9fbd67d118d54d42b36c2095f1fcca6f (127.9.221.130:46369)
I20260811 12:05:52.065021 30861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:52.057092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:40913 (local address 127.9.221.131:42399)
0811 12:05:52.057612 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:52.057634 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:05:52.057765 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:05:52.058999 (+ 1234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:05:52.059005 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:05:52.059031 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:05:52.059224 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:05:52.059236 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:05:52.060749 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:05:52.060754 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:05:52.062873 (+ 2119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:05:52.062887 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:05:52.064292 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:05:52.064298 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:05:52.064482 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:52.064517 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 12:05:52.064604 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 12:05:52.064736 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"mutex_wait_us":1}
I20260811 12:05:52.065747 30879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:52.058270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:42399 (local address 127.9.221.129:40913)
0811 12:05:52.058498 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 12:05:52.058505 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:05:52.058530 (+ 25us) server_negotiation.cc:407] Connection header received
0811 12:05:52.058608 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:05:52.058613 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:05:52.058704 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:05:52.058827 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:05:52.059404 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:52.060571 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:52.063037 (+ 2466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:52.063945 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:52.064107 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:52.064189 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 12:05:52.065364 (+ 1175us) server_negotiation.cc:299] Negotiation successful
0811 12:05:52.065508 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 12:05:52.067173 30880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:52.058051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:59629 (local address 127.9.221.130:46369)
0811 12:05:52.058345 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0811 12:05:52.058351 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:05:52.058822 (+ 471us) server_negotiation.cc:407] Connection header received
0811 12:05:52.059068 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:05:52.059074 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:05:52.059164 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:05:52.059229 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:05:52.060061 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:52.061060 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:52.064375 (+ 3315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:52.065602 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:52.065736 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:52.065831 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 12:05:52.066791 (+ 960us) server_negotiation.cc:299] Negotiation successful
0811 12:05:52.066926 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147}
I20260811 12:05:52.067093 30378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4f329c27da74a15b8e763631ea3ec86" candidate_uuid: "5a10bee5143c47a3863ef6edc6ab7f3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bb16cd74e3944e08501246c3a2b4628" is_pre_election: true
I20260811 12:05:52.067764 30378 raft_consensus.cc:2454] T e4f329c27da74a15b8e763631ea3ec86 P 9bb16cd74e3944e08501246c3a2b4628 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a10bee5143c47a3863ef6edc6ab7f3f in term 0.
I20260811 12:05:52.068118 30913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:52.057558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.130:46369 (local address 127.9.221.131:59629)
0811 12:05:52.058733 (+ 1175us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:52.058751 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:05:52.058918 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:05:52.059441 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:05:52.059461 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:05:52.059492 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:05:52.059880 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:05:52.059895 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:05:52.061257 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:05:52.061262 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:05:52.064184 (+ 2922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:05:52.064199 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:05:52.066022 (+ 1823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:05:52.066028 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:05:52.066209 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:52.066245 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 12:05:52.067569 (+ 1324us) client_negotiation.cc:240] Negotiation successful
0811 12:05:52.067724 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1062,"mutex_wait_us":922,"thread_start_us":5238,"threads_started":1}
I20260811 12:05:52.068473 30696 leader_election.cc:302] T e4f329c27da74a15b8e763631ea3ec86 P 5a10bee5143c47a3863ef6edc6ab7f3f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a10bee5143c47a3863ef6edc6ab7f3f, 9bb16cd74e3944e08501246c3a2b4628; no voters:
I20260811 12:05:52.069044 30897 raft_consensus.cc:2781] T e4f329c27da74a15b8e763631ea3ec86 P 5a10bee5143c47a3863ef6edc6ab7f3f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:05:52.069187 30897 raft_consensus.cc:482] T e4f329c27da74a15b8e763631ea3ec86 P 5a10bee5143c47a3863ef6edc6ab7f3f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:05:52.069303 30897 raft_consensus.cc:3037] T e4f329c27da74a15b8e763631ea3ec86 P 5a10bee5143c47a3863ef6edc6ab7f3f [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:05:52.070000 30594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4f329c27da74a15b8e763631ea3ec86" candidate_uuid: "5a10bee5143c47a3863ef6edc6ab7f3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fbd67d118d54d42b36c2095f1fcca6f" is_pre_election: true
I20260811 12:05:52.070518 30594 raft_consensus.cc:2454] T e4f329c27da74a15b8e763631ea3ec86 P 9fbd67d118d54d42b36c2095f1fcca6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a10bee5143c47a3863ef6edc6ab7f3f in term 0.
I20260811 12:05:52.072307 30897 raft_consensus.cc:504] T e4f329c27da74a15b8e763631ea3ec86 P 5a10bee5143c47a3863ef6edc6ab7f3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bb16cd74e3944e08501246c3a2b4628" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 40913 } } peers { permanent_uuid: "9fbd67d118d54d42b36c2095f1fcca6f" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46369 } } peers { permanent_uuid: "5a10bee5143c47a3863ef6edc6ab7f3f" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43551 } }
I20260811 12:05:52.072960 30897 leader_election.cc:288] T e4f329c27da74a15b8e763631ea3ec86 P 5a10bee5143c47a3863ef6edc6ab7f3f [CANDIDATE]: Term 1 election: Requested vote from peers 9bb16cd74e3944e08501246c3a2b4628 (127.9.221.129:40913), 9fbd67d118d54d42b36c2095f1fcca6f (127.9.221.130:46369)
I20260811 12:05:52.073310 30378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4f329c27da74a15b8e763631ea3ec86" candidate_uuid: "5a10bee5143c47a3863ef6edc6ab7f3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bb16cd74e3944e08501246c3a2b4628"
I20260811 12:05:52.073401 30594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4f329c27da74a15b8e763631ea3ec86" candidate_uuid: "5a10bee5143c47a3863ef6edc6ab7f3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fbd67d118d54d42b36c2095f1fcca6f"
I20260811 12:05:52.073524 30378 raft_consensus.cc:3037] T e4f329c27da74a15b8e763631ea3ec86 P 9bb16cd74e3944e08501246c3a2b4628 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:05:52.073630 30594 raft_consensus.cc:3037] T e4f329c27da74a15b8e763631ea3ec86 P 9fbd67d118d54d42b36c2095f1fcca6f [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:05:52.075898 30594 raft_consensus.cc:2454] T e4f329c27da74a15b8e763631ea3ec86 P 9fbd67d118d54d42b36c2095f1fcca6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a10bee5143c47a3863ef6edc6ab7f3f in term 1.
I20260811 12:05:52.076280 30696 leader_election.cc:302] T e4f329c27da74a15b8e763631ea3ec86 P 5a10bee5143c47a3863ef6edc6ab7f3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a10bee5143c47a3863ef6edc6ab7f3f, 9fbd67d118d54d42b36c2095f1fcca6f; no voters:
I20260811 12:05:52.076560 30897 raft_consensus.cc:2781] T e4f329c27da74a15b8e763631ea3ec86 P 5a10bee5143c47a3863ef6edc6ab7f3f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:05:52.076819 30378 raft_consensus.cc:2454] T e4f329c27da74a15b8e763631ea3ec86 P 9bb16cd74e3944e08501246c3a2b4628 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a10bee5143c47a3863ef6edc6ab7f3f in term 1.
I20260811 12:05:52.077086 30897 raft_consensus.cc:686] T e4f329c27da74a15b8e763631ea3ec86 P 5a10bee5143c47a3863ef6edc6ab7f3f [term 1 LEADER]: Becoming Leader. State: Replica: 5a10bee5143c47a3863ef6edc6ab7f3f, State: Running, Role: LEADER
I20260811 12:05:52.077647 30897 consensus_queue.cc:237] T e4f329c27da74a15b8e763631ea3ec86 P 5a10bee5143c47a3863ef6edc6ab7f3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bb16cd74e3944e08501246c3a2b4628" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 40913 } } peers { permanent_uuid: "9fbd67d118d54d42b36c2095f1fcca6f" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46369 } } peers { permanent_uuid: "5a10bee5143c47a3863ef6edc6ab7f3f" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43551 } }
I20260811 12:05:52.082728 30156 catalog_manager.cc:5495] T e4f329c27da74a15b8e763631ea3ec86 P 5a10bee5143c47a3863ef6edc6ab7f3f reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a10bee5143c47a3863ef6edc6ab7f3f (127.9.221.131). New cstate: current_term: 1 leader_uuid: "5a10bee5143c47a3863ef6edc6ab7f3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bb16cd74e3944e08501246c3a2b4628" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 40913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9fbd67d118d54d42b36c2095f1fcca6f" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a10bee5143c47a3863ef6edc6ab7f3f" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43551 } health_report { overall_health: HEALTHY } } }
I20260811 12:05:52.116945 30883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:52.110472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37526 (local address 127.9.221.131:43551)
0811 12:05:52.110705 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 12:05:52.110712 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:05:52.110746 (+ 34us) server_negotiation.cc:407] Connection header received
0811 12:05:52.110994 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:05:52.110999 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:05:52.111115 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:05:52.111268 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:05:52.111887 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:52.112621 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:52.114714 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:52.115107 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:52.115205 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:52.115835 (+ 630us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:05:52.116352 (+ 517us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:05:52.116474 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 12:05:52.116608 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 12:05:52.116678 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 12:05:52.126238 30594 raft_consensus.cc:1261] T e4f329c27da74a15b8e763631ea3ec86 P 9fbd67d118d54d42b36c2095f1fcca6f [term 1 FOLLOWER]: Refusing update from remote peer 5a10bee5143c47a3863ef6edc6ab7f3f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:05:52.126238 30378 raft_consensus.cc:1261] T e4f329c27da74a15b8e763631ea3ec86 P 9bb16cd74e3944e08501246c3a2b4628 [term 1 FOLLOWER]: Refusing update from remote peer 5a10bee5143c47a3863ef6edc6ab7f3f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:05:52.127314 30914 consensus_queue.cc:1035] T e4f329c27da74a15b8e763631ea3ec86 P 5a10bee5143c47a3863ef6edc6ab7f3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bb16cd74e3944e08501246c3a2b4628" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 40913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:05:52.127743 30897 consensus_queue.cc:1035] T e4f329c27da74a15b8e763631ea3ec86 P 5a10bee5143c47a3863ef6edc6ab7f3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9fbd67d118d54d42b36c2095f1fcca6f" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:05:52.161039 30156 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:05:52.161345 30156 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:05:52.166018 30156 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 540120c3bbe14057a28352ca4e19d1ce: Sending DeleteTablet for 3 replicas of tablet e4f329c27da74a15b8e763631ea3ec86
I20260811 12:05:52.168008 30350 tablet_service.cc:1514] Processing DeleteTablet for tablet e4f329c27da74a15b8e763631ea3ec86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:05:52 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:56268
I20260811 12:05:52.168673 30563 tablet_service.cc:1514] Processing DeleteTablet for tablet e4f329c27da74a15b8e763631ea3ec86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:05:52 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:39690
I20260811 12:05:52.169848 30757 tablet_service.cc:1514] Processing DeleteTablet for tablet e4f329c27da74a15b8e763631ea3ec86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:05:52 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:37514
I20260811 12:05:52.176103 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 30220
I20260811 12:05:52.228268 30930 tablet_replica.cc:330] T e4f329c27da74a15b8e763631ea3ec86 P 9bb16cd74e3944e08501246c3a2b4628: stopping tablet replica
I20260811 12:05:52.229501 30931 tablet_replica.cc:330] T e4f329c27da74a15b8e763631ea3ec86 P 9fbd67d118d54d42b36c2095f1fcca6f: stopping tablet replica
I20260811 12:05:52.230046 30931 raft_consensus.cc:2229] T e4f329c27da74a15b8e763631ea3ec86 P 9fbd67d118d54d42b36c2095f1fcca6f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:05:52.230643 30931 raft_consensus.cc:2258] T e4f329c27da74a15b8e763631ea3ec86 P 9fbd67d118d54d42b36c2095f1fcca6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:05:52.231453 30935 tablet_replica.cc:330] T e4f329c27da74a15b8e763631ea3ec86 P 5a10bee5143c47a3863ef6edc6ab7f3f: stopping tablet replica
I20260811 12:05:52.232162 30935 raft_consensus.cc:2229] T e4f329c27da74a15b8e763631ea3ec86 P 5a10bee5143c47a3863ef6edc6ab7f3f [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:05:52.232990 30935 raft_consensus.cc:2258] T e4f329c27da74a15b8e763631ea3ec86 P 5a10bee5143c47a3863ef6edc6ab7f3f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:05:52.235396 30931 ts_tablet_manager.cc:1905] T e4f329c27da74a15b8e763631ea3ec86 P 9fbd67d118d54d42b36c2095f1fcca6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:05:52.245880 30935 ts_tablet_manager.cc:1905] T e4f329c27da74a15b8e763631ea3ec86 P 5a10bee5143c47a3863ef6edc6ab7f3f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:05:52.247951 30930 raft_consensus.cc:2229] T e4f329c27da74a15b8e763631ea3ec86 P 9bb16cd74e3944e08501246c3a2b4628 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:05:52.249014 30930 raft_consensus.cc:2258] T e4f329c27da74a15b8e763631ea3ec86 P 9bb16cd74e3944e08501246c3a2b4628 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:05:52.254304 30935 ts_tablet_manager.cc:1918] T e4f329c27da74a15b8e763631ea3ec86 P 5a10bee5143c47a3863ef6edc6ab7f3f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:05:52.254559 30935 log.cc:1196] T e4f329c27da74a15b8e763631ea3ec86 P 5a10bee5143c47a3863ef6edc6ab7f3f: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-2/wal/wals/e4f329c27da74a15b8e763631ea3ec86
I20260811 12:05:52.255154 30935 ts_tablet_manager.cc:1939] T e4f329c27da74a15b8e763631ea3ec86 P 5a10bee5143c47a3863ef6edc6ab7f3f: Deleting consensus metadata
I20260811 12:05:52.257032 30138 catalog_manager.cc:4841] TS 5a10bee5143c47a3863ef6edc6ab7f3f (127.9.221.131:43551): tablet e4f329c27da74a15b8e763631ea3ec86 (table test-table [id=69debcf5ffec41adac49efc311bde25e]) successfully deleted
I20260811 12:05:52.259087 30931 ts_tablet_manager.cc:1918] T e4f329c27da74a15b8e763631ea3ec86 P 9fbd67d118d54d42b36c2095f1fcca6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:05:52.259310 30931 log.cc:1196] T e4f329c27da74a15b8e763631ea3ec86 P 9fbd67d118d54d42b36c2095f1fcca6f: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449743084027-10102-0/minicluster-data/ts-1/wal/wals/e4f329c27da74a15b8e763631ea3ec86
I20260811 12:05:52.259984 30931 ts_tablet_manager.cc:1939] T e4f329c27da74a15b8e763631ea3ec86 P 9fbd67d118d54d42b36c2095f1fcca6f: Deleting consensus metadata
I20260811 12:05:52.261804 30138 catalog_manager.cc:4841] TS 9fbd67d118d54d42b36c2095f1fcca6f (127.9.221.130:46369): tablet e4f329c27da74a15b8e763631ea3ec86 (table test-table [id=69debcf5ffec41adac49efc311bde25e]) successfully deleted
I20260811 12:05:52.262297 30930 ts_tablet_manager.cc:1905] T e4f329c27da74a15b8e763631ea3ec86 P 9bb16cd74e3944e08501246c3a2b4628: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:05:52.263520 30879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:52.177493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56270 (local address 127.9.221.129:40913)
0811 12:05:52.177701 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 12:05:52.177707 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:05:52.222581 (+ 44874us) server_negotiation.cc:407] Connection header received
0811 12:05:52.222919 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:05:52.222925 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:05:52.223012 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:05:52.223085 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:05:52.225124 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:52.226357 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:52.239411 (+ 13054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:52.240149 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:52.240268 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:52.250428 (+ 10160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:05:52.250441 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:05:52.250457 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:05:52.250498 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:05:52.254474 (+ 3976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:52.256785 (+ 2311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:52.256794 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:52.256798 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:52.256958 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:52.258461 (+ 1503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:52.258466 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:52.258469 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:52.258966 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:05:52.262145 (+ 3179us) server_negotiation.cc:1035] Waiting for connection context
0811 12:05:52.262833 (+ 688us) server_negotiation.cc:299] Negotiation successful
0811 12:05:52.263091 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260811 12:05:52.265103 30442 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:05:52.359956 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 30220
I20260811 12:05:52.390453 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 30456
W20260811 12:05:52.390352 30138 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.221.129:40913 (error 108)
I20260811 12:05:52.392948 30940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:52.392130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:40913 (local address 127.0.0.1:56274)
0811 12:05:52.392608 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:52.392729 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.129:40913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":227,"threads_started":1}
W20260811 12:05:52.393337 30138 catalog_manager.cc:4584] TS 9bb16cd74e3944e08501246c3a2b4628 (127.9.221.129:40913): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e4f329c27da74a15b8e763631ea3ec86: Network error: Client connection negotiation failed: client connection to 127.9.221.129:40913: connect: Connection refused (error 111)
I20260811 12:05:52.405213 30880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:52.391859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39692 (local address 127.9.221.130:46369)
0811 12:05:52.392068 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 12:05:52.392073 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:05:52.392094 (+ 21us) server_negotiation.cc:407] Connection header received
0811 12:05:52.392158 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:05:52.392163 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:05:52.392251 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:05:52.392383 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:05:52.394146 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:52.394841 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:52.396225 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:52.396632 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:52.396772 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:52.399372 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:05:52.399395 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:05:52.399409 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:05:52.399446 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:05:52.402423 (+ 2977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:52.403124 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:52.403131 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:52.403133 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:52.403270 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:52.403677 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:52.403688 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:52.403691 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:52.404039 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:05:52.404344 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0811 12:05:52.404659 (+ 315us) server_negotiation.cc:299] Negotiation successful
0811 12:05:52.404878 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 12:05:52.405803 30653 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:05:52.447068 30940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:52.446475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:40913 (local address 127.0.0.1:56284)
0811 12:05:52.446755 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:52.446880 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.129:40913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260811 12:05:52.448333 30940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:52.447908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:40913 (local address 127.0.0.1:56300)
0811 12:05:52.448092 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:52.448176 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.129:40913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 12:05:52.496747 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 30456
I20260811 12:05:52.509335 30940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:52.508728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:40913 (local address 127.0.0.1:56308)
0811 12:05:52.508982 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:52.509097 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.129:40913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 12:05:52.510689 30940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:52.510192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:40913 (local address 127.0.0.1:56320)
0811 12:05:52.510402 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:52.510511 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.129:40913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 12:05:52.522504 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 30665
I20260811 12:05:52.538674 30883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:52.523374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37534 (local address 127.9.221.131:43551)
0811 12:05:52.523572 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 12:05:52.523578 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:05:52.523607 (+ 29us) server_negotiation.cc:407] Connection header received
0811 12:05:52.523834 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:05:52.523839 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:05:52.523963 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:05:52.524119 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:05:52.525424 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:52.526693 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:52.528004 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:52.528734 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:52.528886 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:52.531430 (+ 2544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:05:52.531444 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:05:52.531477 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:05:52.531523 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:05:52.535712 (+ 4189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:52.536390 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:52.536399 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:52.536403 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:52.536594 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:52.537062 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:52.537067 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:52.537070 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:52.537549 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:05:52.537800 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0811 12:05:52.538182 (+ 382us) server_negotiation.cc:299] Negotiation successful
0811 12:05:52.538343 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 12:05:52.539374 30852 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:05:52.591188 30940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:52.590660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:40913 (local address 127.0.0.1:56324)
0811 12:05:52.590940 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:52.591040 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.129:40913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260811 12:05:52.592394 30940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:52.591974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:40913 (local address 127.0.0.1:56340)
0811 12:05:52.592192 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:52.592263 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.129:40913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260811 12:05:52.660501 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 30665
I20260811 12:05:52.685633 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 30104
I20260811 12:05:52.686877 30206 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:05:52.800158 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 30104
I20260811 12:05:52.834151 10102 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 12:05:53.180601 10102 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 12:05:53.180831 10102 mini_ranger.cc:66] Stopping Ranger...
I20260811 12:05:53.525959 10102 mini_ranger.cc:68] Stopped Ranger
2026-08-11 12:05:53.526 UTC [28766] LOG: received smart shutdown request
2026-08-11 12:05:53.529 UTC [28766] LOG: background worker "logical replication launcher" (PID 28776) exited with exit code 1
2026-08-11 12:05:53.529 UTC [28770] LOG: shutting down
2026-08-11 12:05:53.665 UTC [28766] LOG: database system is shut down
2026-08-11T12:05:53Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (52563 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:05:53 dist-test-slave-mjjr krb5kdc[30959](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:05:53 dist-test-slave-mjjr krb5kdc[30959](info): set up 2 sockets
Aug 11 12:05:53 dist-test-slave-mjjr krb5kdc[30959](info): commencing operation
krb5kdc: starting...
W20260811 12:05:55.112149 10102 mini_kdc.cc:121] Time spent starting KDC: real 1.265s user 0.004s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Aug 11 12:05:55 dist-test-slave-mjjr krb5kdc[30959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449955, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:05:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:05:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.221.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
I20260811 12:05:55.362747 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:40695
--webserver_interface=127.9.221.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:39011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:40695,127.9.221.189:42629,127.9.221.188:43895
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:05:55.705142 30980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:05:55.705420 30980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:05:55.705510 30980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:05:55.712528 30980 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:05:55.712618 30980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:05:55.712668 30980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:05:55.712764 30980 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:05:55.712802 30980 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:05:55.722215 30980 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:39011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:40695,127.9.221.189:42629,127.9.221.188:43895
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:40695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.221.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:05:55.723429 30980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:05:55.724817 30980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:05:55.736614 30990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:05:55.736621 30994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:05:55.736802 30997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:05:55.737327 30980 server_base.cc:1034] running on GCE node
I20260811 12:05:55.738133 30980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:05:55.739460 30980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:05:55.740662 30980 hybrid_clock.cc:648] HybridClock initialized: now 1786449955740605 us; error 49 us; skew 500 ppm
I20260811 12:05:55.741014 30980 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:05:55 dist-test-slave-mjjr krb5kdc[30959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449955, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:05:55.748874 30980 init.cc:377] Logged in from keytab as kudu/127.9.221.190@KRBTEST.COM (short username kudu)
I20260811 12:05:55.750324 30980 webserver.cc:460] Webserver started at http://127.9.221.190:40547/ using document root <none> and password file <none>
I20260811 12:05:55.751012 30980 fs_manager.cc:359] Metadata directory not provided
I20260811 12:05:55.751179 30980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:05:55.751492 30980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:05:55.754544 30980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "799b4ad904f346869da49c69db52052b"
format_stamp: "Formatted at 2026-08-11 12:05:55 on dist-test-slave-mjjr"
server_key: "b99803d5c9e112b17402480682bd0926"
server_key_iv: "7a9601d3cfdb941a644c994fbc2118cd"
server_key_version: "encryptionkey@0"
I20260811 12:05:55.755376 30980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "799b4ad904f346869da49c69db52052b"
format_stamp: "Formatted at 2026-08-11 12:05:55 on dist-test-slave-mjjr"
server_key: "b99803d5c9e112b17402480682bd0926"
server_key_iv: "7a9601d3cfdb941a644c994fbc2118cd"
server_key_version: "encryptionkey@0"
I20260811 12:05:55.760377 30980 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260811 12:05:55.763749 31008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:05:55.765008 30980 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 12:05:55.765201 30980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "799b4ad904f346869da49c69db52052b"
format_stamp: "Formatted at 2026-08-11 12:05:55 on dist-test-slave-mjjr"
server_key: "b99803d5c9e112b17402480682bd0926"
server_key_iv: "7a9601d3cfdb941a644c994fbc2118cd"
server_key_version: "encryptionkey@0"
I20260811 12:05:55.765393 30980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:05:55.784322 30980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:05:55.788434 30980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:05:55.788725 30980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:05:55.806893 30980 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.190:40695
I20260811 12:05:55.806946 31082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.190:40695 every 8 connection(s)
I20260811 12:05:55.808264 30980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:05:55.811859 31086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:05:55.816375 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 30980
I20260811 12:05:55.816619 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:05:55.817313 10102 external_mini_cluster.cc:1428] Setting key 93b229ffe3cb389b5e28622ca897230c
I20260811 12:05:55.818392 31086 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40695 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:05:55 dist-test-slave-mjjr krb5kdc[30959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449955, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:05:55.840627 31088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:55.821634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:40695 (local address 127.0.0.1:50946)
0811 12:05:55.822387 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:55.822425 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 12:05:55.824567 (+ 2142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:05:55.825378 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:05:55.825384 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:05:55.826081 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:05:55.826409 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:05:55.826425 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:05:55.828041 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:05:55.828046 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:05:55.829102 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:05:55.829113 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:05:55.830029 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:05:55.830036 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:05:55.830231 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:55.830942 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:05:55.830964 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:05:55.833374 (+ 2410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:05:55.836552 (+ 3178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:05:55.836565 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:05:55.836576 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:05:55.836963 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:05:55.837349 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:05:55.837354 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:05:55.837357 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:05:55.837508 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:05:55.838241 (+ 733us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:05:55.838251 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:05:55.838467 (+ 216us) client_negotiation.cc:769] Sending connection context
0811 12:05:55.838712 (+ 245us) client_negotiation.cc:240] Negotiation successful
0811 12:05:55.839334 (+ 622us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"mutex_wait_us":176,"thread_start_us":316,"threads_started":1}
I20260811 12:05:55.840646 31089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:55.822956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50946 (local address 127.9.221.190:40695)
0811 12:05:55.823757 (+ 801us) server_negotiation.cc:206] Beginning negotiation
0811 12:05:55.823765 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:05:55.823824 (+ 59us) server_negotiation.cc:407] Connection header received
0811 12:05:55.824936 (+ 1112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:05:55.824942 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:05:55.825152 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:05:55.825353 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:05:55.826620 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:55.827886 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:55.829242 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:55.829799 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:55.829999 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:55.833594 (+ 3595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:05:55.833601 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:05:55.833611 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:05:55.833638 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:05:55.836414 (+ 2776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:55.837090 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:55.837104 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:55.837107 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:55.837237 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:55.837630 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:55.837637 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:55.837640 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:55.838110 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:05:55.838321 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 12:05:55.838815 (+ 494us) server_negotiation.cc:299] Negotiation successful
0811 12:05:55.839236 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":563,"thread_start_us":384,"threads_started":1}
WARNING: no policy specified for kudu/127.9.221.189@KRBTEST.COM; defaulting to no policy
I20260811 12:05:55.843854 31086 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42629 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:05:55.844997 31088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:55.844546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:42629 (local address 127.0.0.1:47688)
0811 12:05:55.844718 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:55.844838 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.189:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 12:05:55.845302 31010 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.221.189:42629: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.189@KRBTEST.COM" created.
I20260811 12:05:55.847205 31088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:55.846712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:42629 (local address 127.0.0.1:47698)
0811 12:05:55.846918 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:55.847018 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.189:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 12:05:55.847482 31086 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.221.189:42629: Network error: Client connection negotiation failed: client connection to 127.9.221.189:42629: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.189.keytab.
Entry for principal kudu/127.9.221.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.189.keytab.
I20260811 12:05:55.877352 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.189:42629
--webserver_interface=127.9.221.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.189.keytab
--principal=kudu/127.9.221.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:39011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:40695,127.9.221.189:42629,127.9.221.188:43895
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:05:55.896859 31086 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42629 } attempt: 1
I20260811 12:05:55.898114 31088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:55.897655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:42629 (local address 127.0.0.1:47714)
0811 12:05:55.897857 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:55.897984 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.189:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 12:05:55.899116 31088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:55.898766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:42629 (local address 127.0.0.1:47722)
0811 12:05:55.898916 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:55.898992 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.189:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 12:05:55.899300 31086 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.221.189:42629: Network error: Client connection negotiation failed: client connection to 127.9.221.189:42629: connect: Connection refused (error 111)
I20260811 12:05:55.967597 31086 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42629 } attempt: 2
I20260811 12:05:55.968827 31088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:55.968318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:42629 (local address 127.0.0.1:47730)
0811 12:05:55.968595 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:55.968687 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.189:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 12:05:55.969929 31088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:55.969555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:42629 (local address 127.0.0.1:47746)
0811 12:05:55.969704 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:55.969804 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.189:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 12:05:55.970129 31086 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.221.189:42629: Network error: Client connection negotiation failed: client connection to 127.9.221.189:42629: connect: Connection refused (error 111)
I20260811 12:05:56.061492 31086 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42629 } attempt: 3
I20260811 12:05:56.062696 31088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:56.062228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:42629 (local address 127.0.0.1:47758)
0811 12:05:56.062436 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:56.062556 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.189:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 12:05:56.063778 31088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:56.063345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:42629 (local address 127.0.0.1:47766)
0811 12:05:56.063511 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:56.063577 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.189:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 12:05:56.063956 31086 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.221.189:42629: Network error: Client connection negotiation failed: client connection to 127.9.221.189:42629: connect: Connection refused (error 111)
I20260811 12:05:56.207185 31086 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42629 } attempt: 4
I20260811 12:05:56.208436 31088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:56.207879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:42629 (local address 127.0.0.1:47780)
0811 12:05:56.208195 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:56.208283 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.189:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158}
I20260811 12:05:56.209447 31088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:56.209090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:42629 (local address 127.0.0.1:47782)
0811 12:05:56.209237 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:56.209330 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.189:42629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 12:05:56.209641 31086 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.221.189:42629: Network error: Client connection negotiation failed: client connection to 127.9.221.189:42629: connect: Connection refused (error 111)
W20260811 12:05:56.212206 31095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:05:56.212507 31095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:05:56.212656 31095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:05:56.219769 31095 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:05:56.219900 31095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:05:56.220007 31095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:05:56.220112 31095 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:05:56.220202 31095 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:05:56.229574 31095 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:39011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:40695,127.9.221.189:42629,127.9.221.188:43895
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.189.keytab
--principal=kudu/127.9.221.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.189:42629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.221.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:05:56.230857 31095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:05:56.232191 31095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:05:56.243978 31101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:05:56.244041 31103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:05:56.244378 31105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:05:56.245064 31095 server_base.cc:1034] running on GCE node
I20260811 12:05:56.245883 31095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:05:56.247254 31095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:05:56.248474 31095 hybrid_clock.cc:648] HybridClock initialized: now 1786449956248433 us; error 57 us; skew 500 ppm
I20260811 12:05:56.248812 31095 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:05:56 dist-test-slave-mjjr krb5kdc[30959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449956, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:05:56.251953 31095 init.cc:377] Logged in from keytab as kudu/127.9.221.189@KRBTEST.COM (short username kudu)
I20260811 12:05:56.253489 31095 webserver.cc:460] Webserver started at http://127.9.221.189:42437/ using document root <none> and password file <none>
I20260811 12:05:56.254148 31095 fs_manager.cc:359] Metadata directory not provided
I20260811 12:05:56.254299 31095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:05:56.254593 31095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:05:56.257462 31095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-1/data/instance:
uuid: "1daa11d243164431983f2af30050889f"
format_stamp: "Formatted at 2026-08-11 12:05:56 on dist-test-slave-mjjr"
server_key: "378e38ddcb5bb6eeb189e3df11bf69df"
server_key_iv: "9f6cce303f24398c351d7a8364bf76f3"
server_key_version: "encryptionkey@0"
I20260811 12:05:56.258246 31095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-1/wal/instance:
uuid: "1daa11d243164431983f2af30050889f"
format_stamp: "Formatted at 2026-08-11 12:05:56 on dist-test-slave-mjjr"
server_key: "378e38ddcb5bb6eeb189e3df11bf69df"
server_key_iv: "9f6cce303f24398c351d7a8364bf76f3"
server_key_version: "encryptionkey@0"
I20260811 12:05:56.263279 31095 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:05:56.266633 31115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:05:56.268074 31095 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:05:56.268254 31095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-1/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-1/wal
uuid: "1daa11d243164431983f2af30050889f"
format_stamp: "Formatted at 2026-08-11 12:05:56 on dist-test-slave-mjjr"
server_key: "378e38ddcb5bb6eeb189e3df11bf69df"
server_key_iv: "9f6cce303f24398c351d7a8364bf76f3"
server_key_version: "encryptionkey@0"
I20260811 12:05:56.268440 31095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:05:56.290279 31095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:05:56.294415 31095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:05:56.294667 31095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:05:56.312173 31095 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.189:42629
I20260811 12:05:56.312211 31195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.189:42629 every 8 connection(s)
I20260811 12:05:56.313591 31095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-1/data/info.pb
I20260811 12:05:56.317138 31196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:05:56.323875 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 31095
I20260811 12:05:56.323295 31196 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40695 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:05:56.324155 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-1/wal/instance
I20260811 12:05:56.324610 10102 external_mini_cluster.cc:1428] Setting key 1da412f7e1719cc49ba3c9f53b9543f5
Aug 11 12:05:56 dist-test-slave-mjjr krb5kdc[30959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449956, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.189@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:05:56.344839 31089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:56.325914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50962 (local address 127.9.221.190:40695)
0811 12:05:56.326143 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 12:05:56.326149 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:05:56.327368 (+ 1219us) server_negotiation.cc:407] Connection header received
0811 12:05:56.329169 (+ 1801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:05:56.329178 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:05:56.329273 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:05:56.329419 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:05:56.330838 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:56.331973 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:56.333472 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:56.334221 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:56.334366 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:56.337712 (+ 3346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:05:56.337721 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:05:56.337729 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:05:56.337775 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:05:56.341574 (+ 3799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:56.342257 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:56.342265 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:56.342271 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:56.342426 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:56.342822 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:56.342829 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:56.342834 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:56.343275 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:05:56.343438 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 12:05:56.344114 (+ 676us) server_negotiation.cc:299] Negotiation successful
0811 12:05:56.344398 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 12:05:56.345338 31198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:56.326394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:40695 (local address 127.0.0.1:50962)
0811 12:05:56.327181 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:56.327226 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 12:05:56.328906 (+ 1680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:05:56.329620 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:05:56.329626 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:05:56.330109 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:05:56.330642 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:05:56.330658 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:05:56.332129 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:05:56.332133 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:05:56.333323 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:05:56.333335 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:05:56.334440 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:05:56.334445 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:05:56.334623 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:56.335295 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:05:56.335315 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:05:56.337487 (+ 2172us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:05:56.341716 (+ 4229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:05:56.341722 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:05:56.341732 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:05:56.342129 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:05:56.342538 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:05:56.342541 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:05:56.342543 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:05:56.342694 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:05:56.343398 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:05:56.343403 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:05:56.343756 (+ 353us) client_negotiation.cc:769] Sending connection context
0811 12:05:56.343983 (+ 227us) client_negotiation.cc:240] Negotiation successful
0811 12:05:56.344307 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":355,"threads_started":1}
I20260811 12:05:56.347085 31196 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42629 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.9.221.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.188@KRBTEST.COM" created.
Aug 11 12:05:56 dist-test-slave-mjjr krb5kdc[30959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449956, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.189@KRBTEST.COM for kudu/127.9.221.189@KRBTEST.COM
I20260811 12:05:56.360873 31198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:56.347915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:42629 (local address 127.0.0.1:47790)
0811 12:05:56.348069 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:56.348089 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:05:56.348197 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:05:56.348916 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:05:56.348920 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:05:56.349331 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:05:56.349453 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:05:56.349461 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:05:56.350722 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:05:56.350729 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:05:56.351622 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:05:56.351633 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:05:56.352486 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:05:56.352492 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:05:56.352714 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:56.353181 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:05:56.353197 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:05:56.355293 (+ 2096us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:05:56.358329 (+ 3036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:05:56.358336 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:05:56.358341 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:05:56.358737 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:05:56.359089 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:05:56.359094 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:05:56.359098 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:05:56.359245 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:05:56.359863 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:05:56.359870 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:05:56.360027 (+ 157us) client_negotiation.cc:769] Sending connection context
0811 12:05:56.360194 (+ 167us) client_negotiation.cc:240] Negotiation successful
0811 12:05:56.360457 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 12:05:56.360929 31203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:56.348029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47790 (local address 127.9.221.189:42629)
0811 12:05:56.348575 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0811 12:05:56.348580 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:05:56.348602 (+ 22us) server_negotiation.cc:407] Connection header received
0811 12:05:56.348638 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:05:56.348642 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:05:56.348784 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:05:56.348923 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:05:56.349609 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:56.350593 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:56.351800 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:56.352327 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:56.352481 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:56.355504 (+ 3023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:05:56.355514 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:05:56.355525 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:05:56.355553 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:05:56.358201 (+ 2648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:56.358861 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:56.358876 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:56.358878 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:56.358984 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:56.359345 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:56.359352 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:56.359355 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:56.359748 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:05:56.359958 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 12:05:56.360335 (+ 377us) server_negotiation.cc:299] Negotiation successful
0811 12:05:56.360547 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":274,"threads_started":1}
I20260811 12:05:56.363368 31196 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.221.188" port: 43895 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:05:56.364543 31198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:56.364135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:43895 (local address 127.0.0.1:52222)
0811 12:05:56.364276 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:56.364382 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:43895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 12:05:56.364801 31117 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.221.188:43895: connect: Connection refused (error 111)
I20260811 12:05:56.366329 31198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:56.365996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:43895 (local address 127.0.0.1:52232)
0811 12:05:56.366166 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:56.366235 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:43895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 12:05:56.366559 31196 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.221.188:43895: Network error: Client connection negotiation failed: client connection to 127.9.221.188:43895: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.188.keytab.
Entry for principal kudu/127.9.221.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.188.keytab.
I20260811 12:05:56.385219 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.188:43895
--webserver_interface=127.9.221.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.188.keytab
--principal=kudu/127.9.221.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:39011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:40695,127.9.221.189:42629,127.9.221.188:43895
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:05:56.415851 31196 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.221.188" port: 43895 } attempt: 1
I20260811 12:05:56.417006 31198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:56.416547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:43895 (local address 127.0.0.1:52236)
0811 12:05:56.416742 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:56.416851 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:43895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 12:05:56.417954 31198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:56.417628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:43895 (local address 127.0.0.1:52248)
0811 12:05:56.417790 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:56.417851 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:43895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 12:05:56.418176 31196 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.221.188:43895: Network error: Client connection negotiation failed: client connection to 127.9.221.188:43895: connect: Connection refused (error 111)
I20260811 12:05:56.486508 31196 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.221.188" port: 43895 } attempt: 2
I20260811 12:05:56.487703 31198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:56.487185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:43895 (local address 127.0.0.1:52264)
0811 12:05:56.487387 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:56.487490 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:43895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 12:05:56.488741 31198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:56.488409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:43895 (local address 127.0.0.1:52276)
0811 12:05:56.488557 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:56.488633 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:43895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 12:05:56.488987 31196 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.221.188:43895: Network error: Client connection negotiation failed: client connection to 127.9.221.188:43895: connect: Connection refused (error 111)
I20260811 12:05:56.508924 31086 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42629 } attempt: 5
Aug 11 12:05:56 dist-test-slave-mjjr krb5kdc[30959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449955, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for kudu/127.9.221.189@KRBTEST.COM
I20260811 12:05:56.522347 31203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:56.509960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47794 (local address 127.9.221.189:42629)
0811 12:05:56.510153 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 12:05:56.510160 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:05:56.510184 (+ 24us) server_negotiation.cc:407] Connection header received
0811 12:05:56.510256 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:05:56.510260 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:05:56.510341 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:05:56.510460 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:05:56.511259 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:56.512051 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:56.513212 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:56.513675 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:56.513813 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:56.517169 (+ 3356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:05:56.517192 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:05:56.517195 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:05:56.517226 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:05:56.519829 (+ 2603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:56.520475 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:56.520481 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:56.520485 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:56.520603 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:56.520967 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:56.520974 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:56.520977 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:56.521243 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:05:56.521393 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 12:05:56.521778 (+ 385us) server_negotiation.cc:299] Negotiation successful
0811 12:05:56.521994 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 12:05:56.522346 31088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:56.509746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:42629 (local address 127.0.0.1:47794)
0811 12:05:56.509951 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:56.509977 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 12:05:56.510126 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:05:56.510488 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:05:56.510493 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:05:56.510896 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:05:56.511091 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:05:56.511101 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:05:56.512187 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:05:56.512193 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:05:56.513060 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:05:56.513071 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:05:56.513826 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:05:56.513832 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:05:56.514037 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:56.514645 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:05:56.514663 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:05:56.516972 (+ 2309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:05:56.519973 (+ 3001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:05:56.519978 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:05:56.519982 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:05:56.520340 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:05:56.520713 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:05:56.520718 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:05:56.520722 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:05:56.520856 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:05:56.521359 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:05:56.521364 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:05:56.521520 (+ 156us) client_negotiation.cc:769] Sending connection context
0811 12:05:56.521665 (+ 145us) client_negotiation.cc:240] Negotiation successful
0811 12:05:56.521997 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 12:05:56.523698 31086 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.221.188" port: 43895 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:05:56.524731 31088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:56.524366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:43895 (local address 127.0.0.1:52292)
0811 12:05:56.524520 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:56.524605 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:43895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 12:05:56.525707 31088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:56.525404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:43895 (local address 127.0.0.1:52304)
0811 12:05:56.525540 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:56.525621 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:43895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 12:05:56.526003 31086 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.221.188:43895: Network error: Client connection negotiation failed: client connection to 127.9.221.188:43895: connect: Connection refused (error 111)
I20260811 12:05:56.577343 31086 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.221.188" port: 43895 } attempt: 1
I20260811 12:05:56.578913 31088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:56.578108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:43895 (local address 127.0.0.1:52306)
0811 12:05:56.578454 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:56.578635 (+ 181us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:43895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260811 12:05:56.580227 31088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:56.579799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:43895 (local address 127.0.0.1:52320)
0811 12:05:56.579974 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:56.580074 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:43895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 12:05:56.580199 31196 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.221.188" port: 43895 } attempt: 3
W20260811 12:05:56.580528 31086 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.221.188:43895: Network error: Client connection negotiation failed: client connection to 127.9.221.188:43895: connect: Connection refused (error 111)
I20260811 12:05:56.581269 31198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:56.580733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:43895 (local address 127.0.0.1:52332)
0811 12:05:56.580956 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:56.581066 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:43895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260811 12:05:56.582365 31198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:56.581852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:43895 (local address 127.0.0.1:52336)
0811 12:05:56.582100 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:56.582202 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:43895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
W20260811 12:05:56.582590 31196 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.221.188:43895: Network error: Client connection negotiation failed: client connection to 127.9.221.188:43895: connect: Connection refused (error 111)
I20260811 12:05:56.648787 31086 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.221.188" port: 43895 } attempt: 2
I20260811 12:05:56.650027 31088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:56.649536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:43895 (local address 127.0.0.1:52352)
0811 12:05:56.649776 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:56.649865 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:43895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 12:05:56.651103 31088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:56.650810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:43895 (local address 127.0.0.1:52354)
0811 12:05:56.650944 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:56.651007 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:43895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 12:05:56.651360 31086 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.221.188:43895: Network error: Client connection negotiation failed: client connection to 127.9.221.188:43895: connect: Connection refused (error 111)
I20260811 12:05:56.725838 31196 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.221.188" port: 43895 } attempt: 4
I20260811 12:05:56.727190 31198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:56.726566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:43895 (local address 127.0.0.1:52358)
0811 12:05:56.726852 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:56.726975 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:43895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 12:05:56.728396 31198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:56.727973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:43895 (local address 127.0.0.1:52360)
0811 12:05:56.728141 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:56.728247 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:43895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 12:05:56.728647 31196 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.221.188:43895: Network error: Client connection negotiation failed: client connection to 127.9.221.188:43895: connect: Connection refused (error 111)
W20260811 12:05:56.739593 31209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:05:56.739960 31209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:05:56.740127 31209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:05:56.747539 31209 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:05:56.747695 31209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:05:56.747807 31209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:05:56.747920 31209 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:05:56.748006 31209 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:05:56.757607 31086 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.221.188" port: 43895 } attempt: 3
I20260811 12:05:56.758138 31209 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:39011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:40695,127.9.221.189:42629,127.9.221.188:43895
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.188.keytab
--principal=kudu/127.9.221.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.188:43895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.221.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:05:56.758857 31088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:56.758340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:43895 (local address 127.0.0.1:52370)
0811 12:05:56.758611 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:56.758698 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:43895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
I20260811 12:05:56.759863 31209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:05:56.759968 31088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:56.759586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:43895 (local address 127.0.0.1:52380)
0811 12:05:56.759774 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:56.759852 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:43895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
W20260811 12:05:56.760293 31086 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.221.188:43895: Network error: Client connection negotiation failed: client connection to 127.9.221.188:43895: connect: Connection refused (error 111)
I20260811 12:05:56.761353 31209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:05:56.776588 31214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:05:56.776575 31215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:05:56.776796 31220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:05:56.777578 31209 server_base.cc:1034] running on GCE node
I20260811 12:05:56.778313 31209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:05:56.779722 31209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:05:56.781015 31209 hybrid_clock.cc:648] HybridClock initialized: now 1786449956780947 us; error 49 us; skew 500 ppm
I20260811 12:05:56.781383 31209 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:05:56 dist-test-slave-mjjr krb5kdc[30959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449956, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:05:56.784479 31209 init.cc:377] Logged in from keytab as kudu/127.9.221.188@KRBTEST.COM (short username kudu)
I20260811 12:05:56.785872 31209 webserver.cc:460] Webserver started at http://127.9.221.188:36565/ using document root <none> and password file <none>
I20260811 12:05:56.786531 31209 fs_manager.cc:359] Metadata directory not provided
I20260811 12:05:56.786634 31209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:05:56.786883 31209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:05:56.789908 31209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-2/data/instance:
uuid: "d7687bbfd6074fa198cd18b2320bf88d"
format_stamp: "Formatted at 2026-08-11 12:05:56 on dist-test-slave-mjjr"
server_key: "accdf8ed9e7738f549757b517b7d3895"
server_key_iv: "f10abdd2b081c3a363c14b15f66c48b1"
server_key_version: "encryptionkey@0"
I20260811 12:05:56.790663 31209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-2/wal/instance:
uuid: "d7687bbfd6074fa198cd18b2320bf88d"
format_stamp: "Formatted at 2026-08-11 12:05:56 on dist-test-slave-mjjr"
server_key: "accdf8ed9e7738f549757b517b7d3895"
server_key_iv: "f10abdd2b081c3a363c14b15f66c48b1"
server_key_version: "encryptionkey@0"
I20260811 12:05:56.797616 31209 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.004s
I20260811 12:05:56.800830 31229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:05:56.802179 31209 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:05:56.802321 31209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-2/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-2/wal
uuid: "d7687bbfd6074fa198cd18b2320bf88d"
format_stamp: "Formatted at 2026-08-11 12:05:56 on dist-test-slave-mjjr"
server_key: "accdf8ed9e7738f549757b517b7d3895"
server_key_iv: "f10abdd2b081c3a363c14b15f66c48b1"
server_key_version: "encryptionkey@0"
I20260811 12:05:56.802479 31209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:05:56.826411 31209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:05:56.830353 31209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:05:56.830595 31209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:05:56.849295 31209 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.188:43895
I20260811 12:05:56.849359 31306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.188:43895 every 8 connection(s)
I20260811 12:05:56.850630 31209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-2/data/info.pb
I20260811 12:05:56.854151 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 31209
I20260811 12:05:56.854387 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449743084027-10102-0/minicluster-data/master-2/wal/instance
I20260811 12:05:56.854393 31307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:05:56.855046 10102 external_mini_cluster.cc:1428] Setting key 86e7d2c7b45d12df635f517b515712bf
I20260811 12:05:56.863878 31307 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40695 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:05:56 dist-test-slave-mjjr krb5kdc[30959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449955, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.188@KRBTEST.COM
Aug 11 12:05:56 dist-test-slave-mjjr krb5kdc[30959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449956, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.188@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:05:56.879410 31314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:56.856995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52390 (local address 127.9.221.188:43895)
0811 12:05:56.857839 (+ 844us) server_negotiation.cc:206] Beginning negotiation
0811 12:05:56.857849 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:05:56.857889 (+ 40us) server_negotiation.cc:407] Connection header received
0811 12:05:56.859372 (+ 1483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:05:56.859392 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:05:56.859816 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:05:56.860140 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:05:56.861991 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:56.863239 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:56.865150 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:56.865776 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:56.865900 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:56.870847 (+ 4947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:05:56.870862 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:05:56.870873 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:05:56.870913 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:05:56.874890 (+ 3977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:56.875597 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:56.875611 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:56.875616 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:56.875832 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:56.876224 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:56.876229 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:56.876234 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:56.876883 (+ 649us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:05:56.877158 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0811 12:05:56.877685 (+ 527us) server_negotiation.cc:299] Negotiation successful
0811 12:05:56.878062 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":649,"thread_start_us":430,"threads_started":1}
I20260811 12:05:56.882586 31316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:56.866631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:40695 (local address 127.0.0.1:50970)
0811 12:05:56.867415 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:56.867465 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 12:05:56.867597 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:05:56.868313 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:05:56.868320 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:05:56.868890 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:05:56.869143 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:05:56.869151 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:05:56.870399 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:05:56.870404 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:05:56.872023 (+ 1619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:05:56.872038 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:05:56.872866 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:05:56.872875 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:05:56.873078 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:56.873741 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:05:56.873764 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:05:56.876196 (+ 2432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:05:56.879971 (+ 3775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:05:56.879980 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:05:56.879988 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:05:56.880397 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:05:56.880835 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:05:56.880839 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:05:56.880842 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:05:56.881033 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:05:56.881567 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:05:56.881572 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:05:56.881783 (+ 211us) client_negotiation.cc:769] Sending connection context
0811 12:05:56.881924 (+ 141us) client_negotiation.cc:240] Negotiation successful
0811 12:05:56.882181 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":422,"threads_started":1}
I20260811 12:05:56.882753 31315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:56.866612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50970 (local address 127.9.221.190:40695)
0811 12:05:56.868009 (+ 1397us) server_negotiation.cc:206] Beginning negotiation
0811 12:05:56.868014 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:05:56.868029 (+ 15us) server_negotiation.cc:407] Connection header received
0811 12:05:56.868076 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:05:56.868079 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:05:56.868134 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:05:56.868254 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:05:56.869309 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:56.870248 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:56.872172 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:56.872634 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:56.872774 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:56.876375 (+ 3601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:05:56.876381 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:05:56.876385 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:05:56.876421 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:05:56.879837 (+ 3416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:56.880541 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:56.880550 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:56.880555 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:56.880706 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:56.881149 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:56.881152 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:56.881155 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:56.881402 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:05:56.881582 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 12:05:56.882099 (+ 517us) server_negotiation.cc:299] Negotiation successful
0811 12:05:56.882312 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1293,"thread_start_us":1190,"threads_started":1}
I20260811 12:05:56.884359 31307 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42629 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:05:56 dist-test-slave-mjjr krb5kdc[30959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449955, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
Aug 11 12:05:56 dist-test-slave-mjjr krb5kdc[30959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449956, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.188@KRBTEST.COM for kudu/127.9.221.189@KRBTEST.COM
I20260811 12:05:56.898195 31315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:56.882467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50980 (local address 127.9.221.190:40695)
0811 12:05:56.883237 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0811 12:05:56.883242 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:05:56.883257 (+ 15us) server_negotiation.cc:407] Connection header received
0811 12:05:56.883302 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:05:56.883312 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:05:56.883391 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:05:56.883487 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:05:56.885091 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:56.885836 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:56.887110 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:56.887510 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:56.887646 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:56.892228 (+ 4582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:05:56.892233 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:05:56.892236 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:05:56.892263 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:05:56.895549 (+ 3286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:56.896367 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:56.896371 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:56.896373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:56.896482 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:56.896877 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:56.896882 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:56.896886 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:56.897203 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:05:56.897463 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0811 12:05:56.897718 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 12:05:56.897867 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":651,"thread_start_us":506,"threads_started":1}
I20260811 12:05:56.898625 31316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:56.885140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:42629 (local address 127.0.0.1:47810)
0811 12:05:56.885419 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:56.885434 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:05:56.885547 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:05:56.885879 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:05:56.885883 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:05:56.886244 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:05:56.886370 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:05:56.886377 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:05:56.887311 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:05:56.887318 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:05:56.888219 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:05:56.888229 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:05:56.888967 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:05:56.888973 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:05:56.889154 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:56.889700 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:05:56.889715 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:05:56.892363 (+ 2648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:05:56.896003 (+ 3640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:05:56.896009 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:05:56.896015 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:05:56.896499 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:05:56.896911 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:05:56.896914 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:05:56.896916 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:05:56.897050 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:05:56.897657 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:05:56.897661 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:05:56.897820 (+ 159us) client_negotiation.cc:769] Sending connection context
0811 12:05:56.897973 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 12:05:56.898267 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200}
I20260811 12:05:56.898958 31203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:56.885142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47810 (local address 127.9.221.189:42629)
0811 12:05:56.885323 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 12:05:56.885327 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:05:56.885505 (+ 178us) server_negotiation.cc:407] Connection header received
0811 12:05:56.885670 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:05:56.885675 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:05:56.885749 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:05:56.885863 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:05:56.886526 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:56.887180 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:56.888354 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:56.888789 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:56.888934 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:56.892551 (+ 3617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:05:56.892559 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:05:56.892564 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:05:56.892606 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:05:56.895880 (+ 3274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:56.896642 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:56.896645 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:56.896648 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:56.896776 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:56.897161 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:56.897167 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:56.897170 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:56.897499 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:05:56.897670 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 12:05:56.898083 (+ 413us) server_negotiation.cc:299] Negotiation successful
0811 12:05:56.898585 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 12:05:56.899957 31307 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.221.188" port: 43895 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:05:56 dist-test-slave-mjjr krb5kdc[30959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449956, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.188@KRBTEST.COM for kudu/127.9.221.188@KRBTEST.COM
Aug 11 12:05:56 dist-test-slave-mjjr krb5kdc[30959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449955, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.189@KRBTEST.COM
I20260811 12:05:56.913908 31316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:56.900587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:43895 (local address 127.0.0.1:52402)
0811 12:05:56.900742 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:56.900762 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:05:56.900850 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:05:56.901163 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:05:56.901166 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:05:56.901481 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:05:56.901600 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:05:56.901607 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:05:56.902903 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:05:56.902910 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:05:56.904160 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:05:56.904172 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:05:56.905131 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:05:56.905136 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:05:56.905326 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:56.905826 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:05:56.905852 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:05:56.908437 (+ 2585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:05:56.911525 (+ 3088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:05:56.911529 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:05:56.911533 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:05:56.911931 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:05:56.912283 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:05:56.912288 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:05:56.912292 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:05:56.912416 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:05:56.912924 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:05:56.912929 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:05:56.913076 (+ 147us) client_negotiation.cc:769] Sending connection context
0811 12:05:56.913218 (+ 142us) client_negotiation.cc:240] Negotiation successful
0811 12:05:56.913509 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 12:05:56.913997 31314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:56.900705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52402 (local address 127.9.221.188:43895)
0811 12:05:56.900836 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 12:05:56.900840 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:05:56.900853 (+ 13us) server_negotiation.cc:407] Connection header received
0811 12:05:56.900941 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:05:56.900945 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:05:56.901025 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:05:56.901123 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:05:56.901744 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:56.902703 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:56.904309 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:56.904949 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:56.905084 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:56.908629 (+ 3545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:05:56.908637 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:05:56.908644 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:05:56.908683 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:05:56.911364 (+ 2681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:56.912049 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:56.912056 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:56.912060 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:56.912177 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:56.912528 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:56.912532 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:56.912535 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:56.912789 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:05:56.912956 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 12:05:56.913308 (+ 352us) server_negotiation.cc:299] Negotiation successful
0811 12:05:56.913674 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":142,"server-negotiator.queue_time_us":33}
I20260811 12:05:56.916580 31203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:56.899839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47818 (local address 127.9.221.189:42629)
0811 12:05:56.899984 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 12:05:56.899989 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:05:56.900782 (+ 793us) server_negotiation.cc:407] Connection header received
0811 12:05:56.900836 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:05:56.900841 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:05:56.900909 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:05:56.901009 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:05:56.902496 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:56.903606 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:56.905347 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:56.906024 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:56.906160 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:56.911048 (+ 4888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:05:56.911054 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:05:56.911061 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:05:56.911097 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:05:56.913758 (+ 2661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:56.914636 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:56.914644 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:56.914649 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:56.914816 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:56.915255 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:56.915260 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:56.915263 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:56.915552 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:05:56.915871 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0811 12:05:56.916204 (+ 333us) server_negotiation.cc:299] Negotiation successful
0811 12:05:56.916359 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 12:05:56.920217 31307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7687bbfd6074fa198cd18b2320bf88d: Bootstrap starting.
I20260811 12:05:56.921600 10102 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:05:56.923799 31307 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d7687bbfd6074fa198cd18b2320bf88d: Neither blocks nor log segments found. Creating new log.
I20260811 12:05:56.924846 31307 log.cc:824] T 00000000000000000000000000000000 P d7687bbfd6074fa198cd18b2320bf88d: Log is configured to *not* fsync() on all Append() calls
I20260811 12:05:56.928079 31307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7687bbfd6074fa198cd18b2320bf88d: No bootstrap required, opened a new log
I20260811 12:05:56.937489 31086 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.221.188" port: 43895 } attempt: 4
I20260811 12:05:56.941514 31307 raft_consensus.cc:348] T 00000000000000000000000000000000 P d7687bbfd6074fa198cd18b2320bf88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799b4ad904f346869da49c69db52052b" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40695 } } peers { permanent_uuid: "1daa11d243164431983f2af30050889f" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42629 } } peers { permanent_uuid: "d7687bbfd6074fa198cd18b2320bf88d" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 43895 } }
I20260811 12:05:56.942181 31307 raft_consensus.cc:374] T 00000000000000000000000000000000 P d7687bbfd6074fa198cd18b2320bf88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:05:56.942389 31307 raft_consensus.cc:729] T 00000000000000000000000000000000 P d7687bbfd6074fa198cd18b2320bf88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7687bbfd6074fa198cd18b2320bf88d, State: Initialized, Role: FOLLOWER
I20260811 12:05:56.943145 31307 consensus_queue.cc:260] T 00000000000000000000000000000000 P d7687bbfd6074fa198cd18b2320bf88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799b4ad904f346869da49c69db52052b" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40695 } } peers { permanent_uuid: "1daa11d243164431983f2af30050889f" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42629 } } peers { permanent_uuid: "d7687bbfd6074fa198cd18b2320bf88d" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 43895 } }
I20260811 12:05:56.948060 31203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:56.926729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47826 (local address 127.9.221.189:42629)
0811 12:05:56.926901 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 12:05:56.926908 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:05:56.927324 (+ 416us) server_negotiation.cc:407] Connection header received
0811 12:05:56.927630 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:05:56.927659 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:05:56.927756 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:05:56.927843 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:05:56.930870 (+ 3027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:56.932023 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:56.934496 (+ 2473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:56.935182 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:56.935281 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:56.939821 (+ 4540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:05:56.939829 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:05:56.939835 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:05:56.939877 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:05:56.943468 (+ 3591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:56.944773 (+ 1305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:56.944782 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:56.944788 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:56.944982 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:56.945613 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:56.945622 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:56.945627 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:56.946126 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:05:56.946400 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0811 12:05:56.947201 (+ 801us) server_negotiation.cc:299] Negotiation successful
0811 12:05:56.947512 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 12:05:56.950040 31307 sys_catalog.cc:564] T 00000000000000000000000000000000 P d7687bbfd6074fa198cd18b2320bf88d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:05:56.951265 31333 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7687bbfd6074fa198cd18b2320bf88d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799b4ad904f346869da49c69db52052b" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40695 } } peers { permanent_uuid: "1daa11d243164431983f2af30050889f" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42629 } } peers { permanent_uuid: "d7687bbfd6074fa198cd18b2320bf88d" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 43895 } } }
I20260811 12:05:56.951875 31333 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7687bbfd6074fa198cd18b2320bf88d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:05:56.956656 31314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:56.927615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52418 (local address 127.9.221.188:43895)
0811 12:05:56.927855 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 12:05:56.927860 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:05:56.934386 (+ 6526us) server_negotiation.cc:407] Connection header received
0811 12:05:56.934795 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:05:56.934804 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:05:56.934881 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:05:56.934971 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:05:56.937540 (+ 2569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:56.938644 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:56.941319 (+ 2675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:56.942330 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:56.942455 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:56.947043 (+ 4588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:05:56.947050 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:05:56.947054 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:05:56.947092 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:05:56.951116 (+ 4024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:56.952233 (+ 1117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:56.952239 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:56.952244 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:56.952768 (+ 524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:56.953342 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:56.953349 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:56.953355 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:56.954141 (+ 786us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:05:56.954465 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0811 12:05:56.955893 (+ 1428us) server_negotiation.cc:299] Negotiation successful
0811 12:05:56.956183 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260811 12:05:56.961374 31315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:56.926436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50996 (local address 127.9.221.190:40695)
0811 12:05:56.926676 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 12:05:56.926683 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:05:56.930960 (+ 4277us) server_negotiation.cc:407] Connection header received
0811 12:05:56.931121 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:05:56.931130 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:05:56.931234 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:05:56.931340 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:05:56.934479 (+ 3139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:56.935550 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:56.937940 (+ 2390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:56.938608 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:56.938715 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:56.951657 (+ 12942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:05:56.951665 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:05:56.951671 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:05:56.951720 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:05:56.957844 (+ 6124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:56.958787 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:56.958792 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:56.958796 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:56.958984 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:56.959480 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:56.959487 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:56.959490 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:56.959878 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:05:56.960149 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0811 12:05:56.960596 (+ 447us) server_negotiation.cc:299] Negotiation successful
0811 12:05:56.960821 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
Aug 11 12:05:56 dist-test-slave-mjjr krb5kdc[30959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449955, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for kudu/127.9.221.188@KRBTEST.COM
I20260811 12:05:56.972942 31088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:56.938073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:43895 (local address 127.0.0.1:52434)
0811 12:05:56.938469 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:56.938489 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:05:56.938616 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:05:56.956279 (+ 17663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:05:56.956285 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:05:56.956823 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:05:56.957064 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:05:56.957081 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:05:56.958787 (+ 1706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:05:56.958792 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:05:56.960385 (+ 1593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:05:56.960401 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:05:56.961627 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:05:56.961640 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:05:56.961828 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:56.962490 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:05:56.962514 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:05:56.965549 (+ 3035us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:05:56.970028 (+ 4479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:05:56.970033 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:05:56.970037 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:05:56.970524 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:05:56.970994 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:05:56.970999 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:05:56.971003 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:05:56.971162 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:05:56.971953 (+ 791us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:05:56.971961 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:05:56.972144 (+ 183us) client_negotiation.cc:769] Sending connection context
0811 12:05:56.972296 (+ 152us) client_negotiation.cc:240] Negotiation successful
0811 12:05:56.972549 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264}
I20260811 12:05:56.974164 31332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:56.939023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52434 (local address 127.9.221.188:43895)
0811 12:05:56.955924 (+ 16901us) server_negotiation.cc:206] Beginning negotiation
0811 12:05:56.955932 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:05:56.955972 (+ 40us) server_negotiation.cc:407] Connection header received
0811 12:05:56.956023 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:05:56.956027 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:05:56.956104 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:05:56.956225 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:05:56.957276 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:56.958632 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:56.960569 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:56.961246 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:56.961372 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:56.965805 (+ 4433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:05:56.965812 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:05:56.965817 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:05:56.965861 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:05:56.969867 (+ 4006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:56.970664 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:56.970671 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:56.970675 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:56.970850 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:56.971293 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:56.971298 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:56.971302 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:56.971707 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:05:56.971878 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 12:05:56.973348 (+ 1470us) server_negotiation.cc:299] Negotiation successful
0811 12:05:56.973543 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16737,"thread_start_us":17424,"threads_started":1}
W20260811 12:05:56.976156 31352 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d7687bbfd6074fa198cd18b2320bf88d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 12:05:56.976341 31352 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 12:05:56.979921 31086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 799b4ad904f346869da49c69db52052b: Bootstrap starting.
I20260811 12:05:56.967191 31209 master_runner.cc:186] Error getting master registration for 127.9.221.190:40695: OK, instance_id { permanent_uuid: "799b4ad904f346869da49c69db52052b" instance_seqno: 1786449955801706 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 12:05:56.980720 31209 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 12:05:56.983359 31086 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 799b4ad904f346869da49c69db52052b: Neither blocks nor log segments found. Creating new log.
I20260811 12:05:56.984483 31086 log.cc:824] T 00000000000000000000000000000000 P 799b4ad904f346869da49c69db52052b: Log is configured to *not* fsync() on all Append() calls
I20260811 12:05:56.987179 31086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 799b4ad904f346869da49c69db52052b: No bootstrap required, opened a new log
I20260811 12:05:56.995096 31086 raft_consensus.cc:348] T 00000000000000000000000000000000 P 799b4ad904f346869da49c69db52052b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799b4ad904f346869da49c69db52052b" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40695 } } peers { permanent_uuid: "1daa11d243164431983f2af30050889f" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42629 } } peers { permanent_uuid: "d7687bbfd6074fa198cd18b2320bf88d" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 43895 } }
I20260811 12:05:56.995417 31086 raft_consensus.cc:374] T 00000000000000000000000000000000 P 799b4ad904f346869da49c69db52052b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:05:56.995563 31086 raft_consensus.cc:729] T 00000000000000000000000000000000 P 799b4ad904f346869da49c69db52052b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 799b4ad904f346869da49c69db52052b, State: Initialized, Role: FOLLOWER
I20260811 12:05:56.996163 31086 consensus_queue.cc:260] T 00000000000000000000000000000000 P 799b4ad904f346869da49c69db52052b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799b4ad904f346869da49c69db52052b" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40695 } } peers { permanent_uuid: "1daa11d243164431983f2af30050889f" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42629 } } peers { permanent_uuid: "d7687bbfd6074fa198cd18b2320bf88d" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 43895 } }
I20260811 12:05:56.998097 31356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 799b4ad904f346869da49c69db52052b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799b4ad904f346869da49c69db52052b" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40695 } } peers { permanent_uuid: "1daa11d243164431983f2af30050889f" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42629 } } peers { permanent_uuid: "d7687bbfd6074fa198cd18b2320bf88d" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 43895 } } }
I20260811 12:05:56.998572 31356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 799b4ad904f346869da49c69db52052b [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:05:56.999327 31086 sys_catalog.cc:564] T 00000000000000000000000000000000 P 799b4ad904f346869da49c69db52052b [sys.catalog]: configured and running, proceeding with master startup.
W20260811 12:05:57.010587 31371 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 799b4ad904f346869da49c69db52052b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 12:05:57.010767 31371 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 12:05:57.008803 30980 master_runner.cc:186] Error getting master registration for 127.9.221.189:42629: OK, instance_id { permanent_uuid: "1daa11d243164431983f2af30050889f" instance_seqno: 1786449956307306 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 12:05:57.020128 30980 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 12:05:57.027853 31196 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.221.188" port: 43895 } attempt: 5
Aug 11 12:05:57 dist-test-slave-mjjr krb5kdc[30959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449956, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.189@KRBTEST.COM for kudu/127.9.221.188@KRBTEST.COM
W20260811 12:05:57.041348 31015 tablet.cc:2369] T 00000000000000000000000000000000 P 799b4ad904f346869da49c69db52052b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:05:57.046389 31198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:57.028421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:43895 (local address 127.0.0.1:52448)
0811 12:05:57.028764 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0811 12:05:57.028784 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:05:57.028890 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:05:57.029350 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:05:57.029356 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:05:57.029890 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:05:57.030123 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:05:57.030136 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:05:57.031620 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:05:57.031626 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:05:57.033083 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:05:57.033099 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:05:57.034134 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:05:57.034140 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:05:57.034381 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:57.035066 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:05:57.035110 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:05:57.037947 (+ 2837us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:05:57.043517 (+ 5570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:05:57.043522 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:05:57.043527 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:05:57.044081 (+ 554us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:05:57.044534 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:05:57.044540 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:05:57.044544 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:05:57.044706 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:05:57.045373 (+ 667us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:05:57.045380 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:05:57.045535 (+ 155us) client_negotiation.cc:769] Sending connection context
0811 12:05:57.045696 (+ 161us) client_negotiation.cc:240] Negotiation successful
0811 12:05:57.045979 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217}
I20260811 12:05:57.046430 31332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:05:57.028821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52448 (local address 127.9.221.188:43895)
0811 12:05:57.028994 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 12:05:57.029000 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:05:57.029018 (+ 18us) server_negotiation.cc:407] Connection header received
0811 12:05:57.029078 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:05:57.029083 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:05:57.029181 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:05:57.029299 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:05:57.030290 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:57.031461 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:57.033254 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:05:57.033966 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:05:57.034115 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:05:57.039327 (+ 5212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:05:57.039334 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:05:57.039339 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:05:57.039380 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:05:57.043372 (+ 3992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:57.044222 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:57.044228 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:57.044232 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:57.044393 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:05:57.044851 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:05:57.044857 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:05:57.044860 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:05:57.045236 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:05:57.045385 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 12:05:57.045838 (+ 453us) server_negotiation.cc:299] Negotiation successful
0811 12:05:57.046076 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 12:05:57.053934 31196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1daa11d243164431983f2af30050889f: Bootstrap starting.
I20260811 12:05:57.058001 31196 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1daa11d243164431983f2af30050889f: Neither blocks nor log segments found. Creating new log.
I20260811 12:05:57.059333 31196 log.cc:824] T 00000000000000000000000000000000 P 1daa11d243164431983f2af30050889f: Log is configured to *not* fsync() on all Append() calls
I20260811 12:05:57.062525 31196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1daa11d243164431983f2af30050889f: No bootstrap required, opened a new log
I20260811 12:05:57.074968 31196 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1daa11d243164431983f2af30050889f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799b4ad904f346869da49c69db52052b" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40695 } } peers { permanent_uuid: "1daa11d243164431983f2af30050889f" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42629 } } peers { permanent_uuid: "d7687bbfd6074fa198cd18b2320bf88d" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 43895 } }
I20260811 12:05:57.075397 31196 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1daa11d243164431983f2af30050889f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:05:57.075579 31196 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1daa11d243164431983f2af30050889f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1daa11d243164431983f2af30050889f, State: Initialized, Role: FOLLOWER
I20260811 12:05:57.076259 31196 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1daa11d243164431983f2af30050889f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799b4ad904f346869da49c69db52052b" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40695 } } peers { permanent_uuid: "1daa11d243164431983f2af30050889f" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42629 } } peers { permanent_uuid: "d7687bbfd6074fa198cd18b2320bf88d" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 43895 } }
I20260811 12:05:57.077311 31375 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1daa11d243164431983f2af30050889f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799b4ad904f346869da49c69db52052b" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40695 } } peers { permanent_uuid: "1daa11d243164431983f2af30050889f" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42629 } } peers { permanent_uuid: "d7687bbfd6074fa198cd18b2320bf88d" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 43895 } } }
I20260811 12:05:57.077795 31375 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1daa11d243164431983f2af30050889f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:05:57.078667 31196 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1daa11d243164431983f2af30050889f [sys.catalog]: configured and running, proceeding with master startup.
W20260811 12:05:57.082623 31238 tablet.cc:2369] T 00000000000000000000000000000000 P d7687bbfd6074fa198cd18b2320bf88d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:05:57.095623 31095 master_runner.cc:263] No leader master found from master 1daa11d243164431983f2af30050889f
I20260811 12:05:57.095791 31095 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 12:05:57.102670 31390 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1daa11d243164431983f2af30050889f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 12:05:57.102905 31390 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 12:05:57.154994 31333 raft_consensus.cc:482] T 00000000000000000000000000000000 P d7687bbfd6074fa198cd18b2320bf88d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:05:57.155366 31333 raft_consensus.cc:504] T 00000000000000000000000000000000 P d7687bbfd6074fa198cd18b2320bf88d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799b4ad904f346869da49c69db52052b" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40695 } } peers { permanent_uuid: "1daa11d243164431983f2af30050889f" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42629 } } peers { permanent_uuid: "d7687bbfd6074fa198cd18b2320bf88d" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 43895 } }
I20260811 12:05:57.157131 31333 leader_election.cc:288] T 00000000000000000000000000000000 P d7687bbfd6074fa198cd18b2320bf88d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 799b4ad904f346869da49c69db52052b (127.9.221.190:40695), 1daa11d243164431983f2af30050889f (127.9.221.189:42629)
I20260811 12:05:57.158370 31159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d7687bbfd6074fa198cd18b2320bf88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1daa11d243164431983f2af30050889f" is_pre_election: true
I20260811 12:05:57.158991 31159 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1daa11d243164431983f2af30050889f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7687bbfd6074fa198cd18b2320bf88d in term 0.
I20260811 12:05:57.159911 31233 leader_election.cc:302] T 00000000000000000000000000000000 P d7687bbfd6074fa198cd18b2320bf88d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1daa11d243164431983f2af30050889f, d7687bbfd6074fa198cd18b2320bf88d; no voters:
I20260811 12:05:57.161193 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d7687bbfd6074fa198cd18b2320bf88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "799b4ad904f346869da49c69db52052b" is_pre_election: true
I20260811 12:05:57.161792 31045 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 799b4ad904f346869da49c69db52052b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7687bbfd6074fa198cd18b2320bf88d in term 0.
I20260811 12:05:57.162693 31333 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d7687bbfd6074fa198cd18b2320bf88d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:05:57.162842 31333 raft_consensus.cc:482] T 00000000000000000000000000000000 P d7687bbfd6074fa198cd18b2320bf88d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:05:57.162938 31333 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d7687bbfd6074fa198cd18b2320bf88d [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:05:57.166356 31333 raft_consensus.cc:504] T 00000000000000000000000000000000 P d7687bbfd6074fa198cd18b2320bf88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799b4ad904f346869da49c69db52052b" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40695 } } peers { permanent_uuid: "1daa11d243164431983f2af30050889f" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42629 } } peers { permanent_uuid: "d7687bbfd6074fa198cd18b2320bf88d" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 43895 } }
I20260811 12:05:57.167482 31045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d7687bbfd6074fa198cd18b2320bf88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "799b4ad904f346869da49c69db52052b"
I20260811 12:05:57.167771 31045 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 799b4ad904f346869da49c69db52052b [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:05:57.171195 31045 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 799b4ad904f346869da49c69db52052b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7687bbfd6074fa198cd18b2320bf88d in term 1.
I20260811 12:05:57.171810 31235 leader_election.cc:302] T 00000000000000000000000000000000 P d7687bbfd6074fa198cd18b2320bf88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 799b4ad904f346869da49c69db52052b, d7687bbfd6074fa198cd18b2320bf88d; no voters:
I20260811 12:05:57.179342 31159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d7687bbfd6074fa198cd18b2320bf88d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1daa11d243164431983f2af30050889f"
I20260811 12:05:57.179571 31159 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1daa11d243164431983f2af30050889f [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:05:57.183250 31159 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1daa11d243164431983f2af30050889f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7687bbfd6074fa198cd18b2320bf88d in term 1.
I20260811 12:05:57.184008 31333 leader_election.cc:288] T 00000000000000000000000000000000 P d7687bbfd6074fa198cd18b2320bf88d [CANDIDATE]: Term 1 election: Requested vote from peers 799b4ad904f346869da49c69db52052b (127.9.221.190:40695), 1daa11d243164431983f2af30050889f (127.9.221.189:42629)
I20260811 12:05:57.184406 31333 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d7687bbfd6074fa198cd18b2320bf88d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:05:57.184811 31333 raft_consensus.cc:686] T 00000000000000000000000000000000 P d7687bbfd6074fa198cd18b2320bf88d [term 1 LEADER]: Becoming Leader. State: Replica: d7687bbfd6074fa198cd18b2320bf88d, State: Running, Role: LEADER
I20260811 12:05:57.185597 31333 consensus_queue.cc:237] T 00000000000000000000000000000000 P d7687bbfd6074fa198cd18b2320bf88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799b4ad904f346869da49c69db52052b" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40695 } } peers { permanent_uuid: "1daa11d243164431983f2af30050889f" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42629 } } peers { permanent_uuid: "d7687bbfd6074fa198cd18b2320bf88d" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 43895 } }
I20260811 12:05:57.204085 31396 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7687bbfd6074fa198cd18b2320bf88d [sys.catalog]: SysCatalogTable state changed. Reason: New leader d7687bbfd6074fa198cd18b2320bf88d. Latest consensus state: current_term: 1 leader_uuid: "d7687bbfd6074fa198cd18b2320bf88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799b4ad904f346869da49c69db52052b" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40695 } } peers { permanent_uuid: "1daa11d243164431983f2af30050889f" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42629 } } peers { permanent_uuid: "d7687bbfd6074fa198cd18b2320bf88d" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 43895 } } }
I20260811 12:05:57.204639 31396 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7687bbfd6074fa198cd18b2320bf88d [sys.catalog]: This master's current role is: LEADER
I20260811 12:05:57.208838 31398 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:05:57.211203 31398 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:05:57.230875 31159 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1daa11d243164431983f2af30050889f [term 1 FOLLOWER]: Refusing update from remote peer d7687bbfd6074fa198cd18b2320bf88d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:05:57.232019 31396 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d7687bbfd6074fa198cd18b2320bf88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1daa11d243164431983f2af30050889f" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:05:57.239189 31045 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 799b4ad904f346869da49c69db52052b [term 1 FOLLOWER]: Refusing update from remote peer d7687bbfd6074fa198cd18b2320bf88d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:05:57.240518 31396 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d7687bbfd6074fa198cd18b2320bf88d [LEADER]: Connected to new peer: Peer: permanent_uuid: "799b4ad904f346869da49c69db52052b" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:05:57.269273 31356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 799b4ad904f346869da49c69db52052b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d7687bbfd6074fa198cd18b2320bf88d. Latest consensus state: current_term: 1 leader_uuid: "d7687bbfd6074fa198cd18b2320bf88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799b4ad904f346869da49c69db52052b" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40695 } } peers { permanent_uuid: "1daa11d243164431983f2af30050889f" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42629 } } peers { permanent_uuid: "d7687bbfd6074fa198cd18b2320bf88d" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 43895 } } }
I20260811 12:05:57.269637 31356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 799b4ad904f346869da49c69db52052b [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:05:57.280846 31399 mvcc.cc:205] Tried to move back new op lower bound from 7317299024787988480 to 7317299024644284416. Current Snapshot: MvccSnapshot[applied={T|T < 7317299024787988480}]
W20260811 12:05:57.297258 31123 tablet.cc:2369] T 00000000000000000000000000000000 P 1daa11d243164431983f2af30050889f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:05:57.298424 31404 mvcc.cc:205] Tried to move back new op lower bound from 7317299024787988480 to 7317299024644284416. Current Snapshot: MvccSnapshot[applied={T|T < 7317299024787988480}]
I20260811 12:05:57.305830 31333 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7687bbfd6074fa198cd18b2320bf88d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d7687bbfd6074fa198cd18b2320bf88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799b4ad904f346869da49c69db52052b" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40695 } } peers { permanent_uuid: "1daa11d243164431983f2af30050889f" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42629 } } peers { permanent_uuid: "d7687bbfd6074fa198cd18b2320bf88d" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 43895 } } }
I20260811 12:05:57.306216 31333 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7687bbfd6074fa198cd18b2320bf88d [sys.catalog]: This master's current role is: LEADER
I20260811 12:05:57.311028 31398 catalog_manager.cc:1347] Generated new cluster ID: f2fcf1db6a454ac59eda54fca4ba533a
I20260811 12:05:57.311131 31398 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:05:57.344841 31356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 799b4ad904f346869da49c69db52052b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d7687bbfd6074fa198cd18b2320bf88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799b4ad904f346869da49c69db52052b" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40695 } } peers { permanent_uuid: "1daa11d243164431983f2af30050889f" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42629 } } peers { permanent_uuid: "d7687bbfd6074fa198cd18b2320bf88d" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 43895 } } }
I20260811 12:05:57.345170 31356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 799b4ad904f346869da49c69db52052b [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:05:57.348251 31375 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1daa11d243164431983f2af30050889f [sys.catalog]: SysCatalogTable state changed. Reason: New leader d7687bbfd6074fa198cd18b2320bf88d. Latest consensus state: current_term: 1 leader_uuid: "d7687bbfd6074fa198cd18b2320bf88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799b4ad904f346869da49c69db52052b" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40695 } } peers { permanent_uuid: "1daa11d243164431983f2af30050889f" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42629 } } peers { permanent_uuid: "d7687bbfd6074fa198cd18b2320bf88d" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 43895 } } }
I20260811 12:05:57.348577 31375 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1daa11d243164431983f2af30050889f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:05:57.358691 31396 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7687bbfd6074fa198cd18b2320bf88d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d7687bbfd6074fa198cd18b2320bf88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799b4ad904f346869da49c69db52052b" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40695 } } peers { permanent_uuid: "1daa11d243164431983f2af30050889f" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42629 } } peers { permanent_uuid: "d7687bbfd6074fa198cd18b2320bf88d" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 43895 } } }
I20260811 12:05:57.360133 31396 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7687bbfd6074fa198cd18b2320bf88d [sys.catalog]: This master's current role is: LEADER
I20260811 12:05:57.366644 31375 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1daa11d243164431983f2af30050889f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d7687bbfd6074fa198cd18b2320bf88d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "799b4ad904f346869da49c69db52052b" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40695 } } peers { permanent_uuid: "1daa11d243164431983f2af30050889f" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42629 } } peers { permanent_uuid: "d7687bbfd6074fa198cd18b2320bf88d" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 43895 } } }
I20260811 12:05:57.366895 31375 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1daa11d243164431983f2af30050889f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:05:57.401777 31398 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:05:57.404166 31398 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:05:57.437278 31398 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d7687bbfd6074fa198cd18b2320bf88d: Generated new TSK 0
I20260811 12:05:57.438546 31398 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:05:58.012782 31371 catalog_manager.cc:1259] Loaded cluster ID: f2fcf1db6a454ac59eda54fca4ba533a
I20260811 12:05:58.012887 31371 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 799b4ad904f346869da49c69db52052b: loading cluster ID for follower catalog manager: success
I20260811 12:05:58.015277 31371 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 799b4ad904f346869da49c69db52052b: acquiring CA information for follower catalog manager: success
I20260811 12:05:58.017036 31371 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 799b4ad904f346869da49c69db52052b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 12:05:58.105065 31390 catalog_manager.cc:1259] Loaded cluster ID: f2fcf1db6a454ac59eda54fca4ba533a
I20260811 12:05:58.105176 31390 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1daa11d243164431983f2af30050889f: loading cluster ID for follower catalog manager: success
I20260811 12:05:58.107776 31390 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1daa11d243164431983f2af30050889f: acquiring CA information for follower catalog manager: success
I20260811 12:05:58.109479 31390 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1daa11d243164431983f2af30050889f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 12:05:58.580788 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 30980
I20260811 12:05:58.582233 31074 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:05:58.694978 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 30980
I20260811 12:05:58.725915 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 31095
I20260811 12:05:58.727022 31190 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:05:58.838824 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 31095
I20260811 12:05:58.868248 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 31209
I20260811 12:05:58.869304 31301 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:05:58.977348 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 31209
2026-08-11T12:05:59Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (5180 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449743084027-10102-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:05:59 dist-test-slave-mjjr krb5kdc[31427](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:05:59 dist-test-slave-mjjr krb5kdc[31427](info): set up 2 sockets
Aug 11 12:05:59 dist-test-slave-mjjr krb5kdc[31427](info): commencing operation
krb5kdc: starting...
W20260811 12:06:00.319888 10102 mini_kdc.cc:121] Time spent starting KDC: real 1.293s user 0.000s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Aug 11 12:06:00 dist-test-slave-mjjr krb5kdc[31427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449960, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:06:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:06:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.221.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
I20260811 12:06:00.578966 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:33859
--webserver_interface=127.9.221.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:46715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:33859
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:06:00.918156 31445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:06:00.918445 31445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:06:00.918649 31445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:06:00.925864 31445 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:06:00.925969 31445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:06:00.926103 31445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:06:00.926204 31445 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:06:00.926290 31445 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:06:00.936561 31445 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:46715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:33859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:33859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.221.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:06:00.938038 31445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:06:00.939474 31445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:06:00.951303 31454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:06:00.951362 31450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:06:00.951362 31456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:06:00.952580 31445 server_base.cc:1034] running on GCE node
I20260811 12:06:00.953293 31445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:06:00.954629 31445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:06:00.955839 31445 hybrid_clock.cc:648] HybridClock initialized: now 1786449960955794 us; error 31 us; skew 500 ppm
I20260811 12:06:00.956192 31445 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:06:00 dist-test-slave-mjjr krb5kdc[31427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449960, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:06:00.963949 31445 init.cc:377] Logged in from keytab as kudu/127.9.221.190@KRBTEST.COM (short username kudu)
I20260811 12:06:00.965517 31445 webserver.cc:460] Webserver started at http://127.9.221.190:39147/ using document root <none> and password file <none>
I20260811 12:06:00.966223 31445 fs_manager.cc:359] Metadata directory not provided
I20260811 12:06:00.966365 31445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:06:00.966707 31445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:06:00.969774 31445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "48d7e4f8c00a4ae7b957f913513ba726"
format_stamp: "Formatted at 2026-08-11 12:06:00 on dist-test-slave-mjjr"
server_key: "05eec3f6c736660c54698da03fcf6dfe"
server_key_iv: "f5a9fb7134bfb19a285a894fb60801a3"
server_key_version: "encryptionkey@0"
I20260811 12:06:00.970528 31445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "48d7e4f8c00a4ae7b957f913513ba726"
format_stamp: "Formatted at 2026-08-11 12:06:00 on dist-test-slave-mjjr"
server_key: "05eec3f6c736660c54698da03fcf6dfe"
server_key_iv: "f5a9fb7134bfb19a285a894fb60801a3"
server_key_version: "encryptionkey@0"
I20260811 12:06:00.975607 31445 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 12:06:00.978893 31472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:06:00.980196 31445 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:06:00.980389 31445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "48d7e4f8c00a4ae7b957f913513ba726"
format_stamp: "Formatted at 2026-08-11 12:06:00 on dist-test-slave-mjjr"
server_key: "05eec3f6c736660c54698da03fcf6dfe"
server_key_iv: "f5a9fb7134bfb19a285a894fb60801a3"
server_key_version: "encryptionkey@0"
I20260811 12:06:00.980576 31445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:06:00.995121 31445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:06:00.999224 31445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:06:00.999516 31445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:06:01.018253 31445 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.190:33859
I20260811 12:06:01.018272 31545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.190:33859 every 8 connection(s)
I20260811 12:06:01.019620 31445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:06:01.023160 31546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:06:01.025076 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 31445
I20260811 12:06:01.025323 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:06:01.026194 10102 external_mini_cluster.cc:1428] Setting key 2fc4e9dced1c4c267e43a78a15e547d4
I20260811 12:06:01.036055 31546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48d7e4f8c00a4ae7b957f913513ba726: Bootstrap starting.
I20260811 12:06:01.038908 31546 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 48d7e4f8c00a4ae7b957f913513ba726: Neither blocks nor log segments found. Creating new log.
I20260811 12:06:01.040081 31546 log.cc:824] T 00000000000000000000000000000000 P 48d7e4f8c00a4ae7b957f913513ba726: Log is configured to *not* fsync() on all Append() calls
Aug 11 12:06:01 dist-test-slave-mjjr krb5kdc[31427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449960, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:06:01.042760 31546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48d7e4f8c00a4ae7b957f913513ba726: No bootstrap required, opened a new log
I20260811 12:06:01.048738 31552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:01.028131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45454 (local address 127.9.221.190:33859)
0811 12:06:01.028909 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:01.028920 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:01.028974 (+ 54us) server_negotiation.cc:407] Connection header received
0811 12:06:01.030614 (+ 1640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:01.030636 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:01.031025 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:01.031390 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:06:01.033029 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:01.034420 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:01.035813 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:01.036568 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:01.036795 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:01.041274 (+ 4479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:06:01.041281 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:06:01.041294 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:06:01.041334 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:06:01.044545 (+ 3211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:01.045320 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:01.045335 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:01.045341 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:01.045579 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:01.046013 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:01.046017 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:01.046019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:01.046525 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:06:01.046862 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:01.047330 (+ 468us) server_negotiation.cc:299] Negotiation successful
0811 12:06:01.047780 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":330,"threads_started":1}
I20260811 12:06:01.050864 31546 raft_consensus.cc:348] T 00000000000000000000000000000000 P 48d7e4f8c00a4ae7b957f913513ba726 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48d7e4f8c00a4ae7b957f913513ba726" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 33859 } }
I20260811 12:06:01.051321 31546 raft_consensus.cc:374] T 00000000000000000000000000000000 P 48d7e4f8c00a4ae7b957f913513ba726 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:06:01.051442 31546 raft_consensus.cc:729] T 00000000000000000000000000000000 P 48d7e4f8c00a4ae7b957f913513ba726 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48d7e4f8c00a4ae7b957f913513ba726, State: Initialized, Role: FOLLOWER
I20260811 12:06:01.052053 31546 consensus_queue.cc:260] T 00000000000000000000000000000000 P 48d7e4f8c00a4ae7b957f913513ba726 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48d7e4f8c00a4ae7b957f913513ba726" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 33859 } }
I20260811 12:06:01.052364 31546 raft_consensus.cc:388] T 00000000000000000000000000000000 P 48d7e4f8c00a4ae7b957f913513ba726 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:06:01.052515 31546 raft_consensus.cc:482] T 00000000000000000000000000000000 P 48d7e4f8c00a4ae7b957f913513ba726 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:06:01.052714 31546 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 48d7e4f8c00a4ae7b957f913513ba726 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:06:01.052922 10102 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:06:01.054914 31546 raft_consensus.cc:504] T 00000000000000000000000000000000 P 48d7e4f8c00a4ae7b957f913513ba726 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48d7e4f8c00a4ae7b957f913513ba726" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 33859 } }
I20260811 12:06:01.055558 31546 leader_election.cc:302] T 00000000000000000000000000000000 P 48d7e4f8c00a4ae7b957f913513ba726 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 48d7e4f8c00a4ae7b957f913513ba726; no voters:
I20260811 12:06:01.056257 31546 leader_election.cc:288] T 00000000000000000000000000000000 P 48d7e4f8c00a4ae7b957f913513ba726 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:06:01.056341 31554 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 48d7e4f8c00a4ae7b957f913513ba726 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:06:01.057057 31554 raft_consensus.cc:686] T 00000000000000000000000000000000 P 48d7e4f8c00a4ae7b957f913513ba726 [term 1 LEADER]: Becoming Leader. State: Replica: 48d7e4f8c00a4ae7b957f913513ba726, State: Running, Role: LEADER
I20260811 12:06:01.057695 31554 consensus_queue.cc:237] T 00000000000000000000000000000000 P 48d7e4f8c00a4ae7b957f913513ba726 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48d7e4f8c00a4ae7b957f913513ba726" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 33859 } }
I20260811 12:06:01.058494 31546 sys_catalog.cc:564] T 00000000000000000000000000000000 P 48d7e4f8c00a4ae7b957f913513ba726 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:06:01.060608 31555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48d7e4f8c00a4ae7b957f913513ba726 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "48d7e4f8c00a4ae7b957f913513ba726" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48d7e4f8c00a4ae7b957f913513ba726" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 33859 } } }
I20260811 12:06:01.060683 31556 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48d7e4f8c00a4ae7b957f913513ba726 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 48d7e4f8c00a4ae7b957f913513ba726. Latest consensus state: current_term: 1 leader_uuid: "48d7e4f8c00a4ae7b957f913513ba726" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48d7e4f8c00a4ae7b957f913513ba726" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 33859 } } }
I20260811 12:06:01.061008 31555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48d7e4f8c00a4ae7b957f913513ba726 [sys.catalog]: This master's current role is: LEADER
I20260811 12:06:01.061076 31556 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48d7e4f8c00a4ae7b957f913513ba726 [sys.catalog]: This master's current role is: LEADER
I20260811 12:06:01.063257 31563 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:06:01.069264 31563 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:06:01.077142 31563 catalog_manager.cc:1347] Generated new cluster ID: 7ea5f80eb4ef449d801043d94bfd5fb5
I20260811 12:06:01.077250 31563 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:06:01.106869 31563 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:06:01.108423 31563 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:06:01.117293 31563 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 48d7e4f8c00a4ae7b957f913513ba726: Generated new TSK 0
I20260811 12:06:01.118343 31563 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:06:01.166517 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 31445
I20260811 12:06:01.168084 31539 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:06:01.265236 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 31445
2026-08-11T12:06:01Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2284 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:06:01 dist-test-slave-mjjr krb5kdc[31590](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:06:01 dist-test-slave-mjjr krb5kdc[31590](info): set up 2 sockets
Aug 11 12:06:01 dist-test-slave-mjjr krb5kdc[31590](info): commencing operation
krb5kdc: starting...
W20260811 12:06:02.608608 10102 mini_kdc.cc:121] Time spent starting KDC: real 1.297s user 0.001s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Aug 11 12:06:02 dist-test-slave-mjjr krb5kdc[31590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:06:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:06:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.221.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
I20260811 12:06:02.876888 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:32893
--webserver_interface=127.9.221.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:37169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:32893
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:06:03.208622 31608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:06:03.208907 31608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:06:03.209056 31608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:06:03.216285 31608 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:06:03.216382 31608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:06:03.216504 31608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:06:03.216609 31608 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:06:03.216672 31608 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:06:03.226303 31608 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:37169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:32893
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:32893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.221.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:06:03.227576 31608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:06:03.228916 31608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:06:03.240881 31618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:06:03.240968 31623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:06:03.240931 31621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:06:03.241833 31608 server_base.cc:1034] running on GCE node
I20260811 12:06:03.242576 31608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:06:03.243990 31608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:06:03.245216 31608 hybrid_clock.cc:648] HybridClock initialized: now 1786449963245178 us; error 50 us; skew 500 ppm
I20260811 12:06:03.245574 31608 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:06:03 dist-test-slave-mjjr krb5kdc[31590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449963, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:06:03.248682 31608 init.cc:377] Logged in from keytab as kudu/127.9.221.190@KRBTEST.COM (short username kudu)
I20260811 12:06:03.250192 31608 webserver.cc:460] Webserver started at http://127.9.221.190:41771/ using document root <none> and password file <none>
I20260811 12:06:03.250866 31608 fs_manager.cc:359] Metadata directory not provided
I20260811 12:06:03.251032 31608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:06:03.251362 31608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:06:03.254299 31608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "b26f2b577ada48839a0741a87dd73acc"
format_stamp: "Formatted at 2026-08-11 12:06:03 on dist-test-slave-mjjr"
server_key: "5a61fb635d34e6b1e61380c5b17ea827"
server_key_iv: "a76de3da28dd8119ae7c728dd9eec038"
server_key_version: "encryptionkey@0"
I20260811 12:06:03.255107 31608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "b26f2b577ada48839a0741a87dd73acc"
format_stamp: "Formatted at 2026-08-11 12:06:03 on dist-test-slave-mjjr"
server_key: "5a61fb635d34e6b1e61380c5b17ea827"
server_key_iv: "a76de3da28dd8119ae7c728dd9eec038"
server_key_version: "encryptionkey@0"
I20260811 12:06:03.260576 31608 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260811 12:06:03.263922 31638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:06:03.265162 31608 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 12:06:03.265341 31608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "b26f2b577ada48839a0741a87dd73acc"
format_stamp: "Formatted at 2026-08-11 12:06:03 on dist-test-slave-mjjr"
server_key: "5a61fb635d34e6b1e61380c5b17ea827"
server_key_iv: "a76de3da28dd8119ae7c728dd9eec038"
server_key_version: "encryptionkey@0"
I20260811 12:06:03.265539 31608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:06:03.288275 31608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:06:03.292196 31608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:06:03.292491 31608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:06:03.311520 31608 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.190:32893
I20260811 12:06:03.311520 31714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.190:32893 every 8 connection(s)
I20260811 12:06:03.312904 31608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:06:03.316429 31715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:06:03.321087 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 31608
I20260811 12:06:03.321316 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:06:03.321996 10102 external_mini_cluster.cc:1428] Setting key 704bd149771ecc9bcc39aaef9b54820d
I20260811 12:06:03.328301 31715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc: Bootstrap starting.
I20260811 12:06:03.332067 31715 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc: Neither blocks nor log segments found. Creating new log.
I20260811 12:06:03.333036 31715 log.cc:824] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc: Log is configured to *not* fsync() on all Append() calls
I20260811 12:06:03.335888 31715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc: No bootstrap required, opened a new log
Aug 11 12:06:03 dist-test-slave-mjjr krb5kdc[31590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:06:03.346174 31715 raft_consensus.cc:348] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b26f2b577ada48839a0741a87dd73acc" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 32893 } }
I20260811 12:06:03.346648 31715 raft_consensus.cc:374] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:06:03.346791 31715 raft_consensus.cc:729] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b26f2b577ada48839a0741a87dd73acc, State: Initialized, Role: FOLLOWER
I20260811 12:06:03.347193 31718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:03.324096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41532 (local address 127.9.221.190:32893)
0811 12:06:03.325014 (+ 918us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:03.325032 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:03.325080 (+ 48us) server_negotiation.cc:407] Connection header received
0811 12:06:03.327412 (+ 2332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:03.327450 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:03.327932 (+ 482us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:03.328334 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:06:03.329680 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:03.331827 (+ 2147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:03.333274 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:03.333975 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:03.334209 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:03.338515 (+ 4306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:06:03.338525 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:06:03.338541 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:06:03.338587 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:06:03.342589 (+ 4002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:03.343313 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:03.343321 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:03.343335 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:03.343588 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:03.343991 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:03.344001 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:03.344004 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:03.344701 (+ 697us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:06:03.345021 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:03.345363 (+ 342us) server_negotiation.cc:299] Negotiation successful
0811 12:06:03.345828 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":396,"threads_started":1}
I20260811 12:06:03.347450 31715 consensus_queue.cc:260] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b26f2b577ada48839a0741a87dd73acc" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 32893 } }
I20260811 12:06:03.347841 31715 raft_consensus.cc:388] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:06:03.348031 31715 raft_consensus.cc:482] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:06:03.348220 31715 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:06:03.350471 31715 raft_consensus.cc:504] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b26f2b577ada48839a0741a87dd73acc" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 32893 } }
I20260811 12:06:03.350944 31715 leader_election.cc:302] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b26f2b577ada48839a0741a87dd73acc; no voters:
I20260811 12:06:03.351505 31715 leader_election.cc:288] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:06:03.351636 31723 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:06:03.352392 31723 raft_consensus.cc:686] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [term 1 LEADER]: Becoming Leader. State: Replica: b26f2b577ada48839a0741a87dd73acc, State: Running, Role: LEADER
I20260811 12:06:03.352929 31723 consensus_queue.cc:237] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b26f2b577ada48839a0741a87dd73acc" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 32893 } }
I20260811 12:06:03.353564 31715 sys_catalog.cc:564] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:06:03.355677 31725 sys_catalog.cc:455] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [sys.catalog]: SysCatalogTable state changed. Reason: New leader b26f2b577ada48839a0741a87dd73acc. Latest consensus state: current_term: 1 leader_uuid: "b26f2b577ada48839a0741a87dd73acc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b26f2b577ada48839a0741a87dd73acc" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 32893 } } }
I20260811 12:06:03.356078 31725 sys_catalog.cc:458] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [sys.catalog]: This master's current role is: LEADER
I20260811 12:06:03.357688 31729 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:06:03.357642 31724 sys_catalog.cc:455] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b26f2b577ada48839a0741a87dd73acc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b26f2b577ada48839a0741a87dd73acc" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 32893 } } }
I20260811 12:06:03.358098 31724 sys_catalog.cc:458] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [sys.catalog]: This master's current role is: LEADER
I20260811 12:06:03.363399 31729 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:06:03.372996 31729 catalog_manager.cc:1347] Generated new cluster ID: 0441414dd2804a6984a06b101f730a8c
I20260811 12:06:03.373111 31729 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:06:03.395416 31729 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:06:03.396687 31729 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:06:03.406042 31729 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc: Generated new TSK 0
I20260811 12:06:03.406803 31729 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.221.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab.
Entry for principal kudu/127.9.221.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab.
I20260811 12:06:03.476212 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.129:0
--local_ip_for_outbound_sockets=127.9.221.129
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:32893
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab
--principal=kudu/127.9.221.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:37169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:06:03.809305 31755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:06:03.809556 31755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:06:03.809755 31755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:06:03.817138 31755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:06:03.817269 31755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:06:03.817469 31755 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.129
I20260811 12:06:03.827359 31755 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:37169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab
--principal=kudu/127.9.221.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:32893
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.129
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:06:03.828694 31755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:06:03.830171 31755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:06:03.843863 31767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:06:03.843935 31762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:06:03.843902 31764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:06:03.844717 31755 server_base.cc:1034] running on GCE node
I20260811 12:06:03.845273 31755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:06:03.846272 31755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:06:03.847492 31755 hybrid_clock.cc:648] HybridClock initialized: now 1786449963847446 us; error 40 us; skew 500 ppm
I20260811 12:06:03.847815 31755 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:06:03 dist-test-slave-mjjr krb5kdc[31590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449963, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:06:03.851068 31755 init.cc:377] Logged in from keytab as kudu/127.9.221.129@KRBTEST.COM (short username kudu)
I20260811 12:06:03.852499 31755 webserver.cc:460] Webserver started at http://127.9.221.129:43271/ using document root <none> and password file <none>
I20260811 12:06:03.853183 31755 fs_manager.cc:359] Metadata directory not provided
I20260811 12:06:03.853327 31755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:06:03.853695 31755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:06:03.856593 31755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-0/data/instance:
uuid: "380d1e238e15456d98dee6b99cdf4f78"
format_stamp: "Formatted at 2026-08-11 12:06:03 on dist-test-slave-mjjr"
server_key: "aedf699819165298ab6f5baee3205207"
server_key_iv: "1ea069fc7f6394608d75adb016ee9668"
server_key_version: "encryptionkey@0"
I20260811 12:06:03.857419 31755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance:
uuid: "380d1e238e15456d98dee6b99cdf4f78"
format_stamp: "Formatted at 2026-08-11 12:06:03 on dist-test-slave-mjjr"
server_key: "aedf699819165298ab6f5baee3205207"
server_key_iv: "1ea069fc7f6394608d75adb016ee9668"
server_key_version: "encryptionkey@0"
I20260811 12:06:03.862156 31755 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 12:06:03.865929 31775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:06:03.867834 31755 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:06:03.868007 31755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-0/wal
uuid: "380d1e238e15456d98dee6b99cdf4f78"
format_stamp: "Formatted at 2026-08-11 12:06:03 on dist-test-slave-mjjr"
server_key: "aedf699819165298ab6f5baee3205207"
server_key_iv: "1ea069fc7f6394608d75adb016ee9668"
server_key_version: "encryptionkey@0"
I20260811 12:06:03.868240 31755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:06:03.886987 31755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:06:03.890494 31755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:06:03.890724 31755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:06:03.891783 31755 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:06:03.893438 31755 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:06:03.893510 31755 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:06:03.893595 31755 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:06:03.893639 31755 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:06:03.928100 31755 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.129:43519
I20260811 12:06:03.928148 31935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.129:43519 every 8 connection(s)
I20260811 12:06:03.929397 31755 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
I20260811 12:06:03.932226 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 31755
I20260811 12:06:03.932482 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance
I20260811 12:06:03.933282 10102 external_mini_cluster.cc:1428] Setting key 84f543b2333c78b281457184c90a782d
Aug 11 12:06:03 dist-test-slave-mjjr krb5kdc[31590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449963, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:06:03.954387 31939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:03.932079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:38911 (local address 127.9.221.190:32893)
0811 12:06:03.932575 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:03.932582 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:03.933754 (+ 1172us) server_negotiation.cc:407] Connection header received
0811 12:06:03.935702 (+ 1948us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:03.935706 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:03.935770 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:03.935859 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:06:03.938041 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:03.938830 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:03.941481 (+ 2651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:03.941892 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:03.942025 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:03.947079 (+ 5054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:06:03.947084 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:06:03.947088 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:06:03.947124 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:06:03.950822 (+ 3698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:03.951634 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:03.951669 (+ 35us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:03.951676 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:03.951854 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:03.952289 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:03.952297 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:03.952303 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:03.952693 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:06:03.952847 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:03.953584 (+ 737us) server_negotiation.cc:299] Negotiation successful
0811 12:06:03.953881 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":246,"threads_started":1}
I20260811 12:06:03.955276 31941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:03.932662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.129:38911)
0811 12:06:03.933557 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:03.933607 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 12:06:03.935355 (+ 1748us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:06:03.936160 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:06:03.936173 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:06:03.936972 (+ 799us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:06:03.937811 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:03.937840 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:03.939035 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:03.939043 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:03.941289 (+ 2246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:03.941307 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:03.942172 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:03.942182 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:03.942509 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:03.943339 (+ 830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:06:03.943374 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:06:03.946794 (+ 3420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:06:03.951000 (+ 4206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:06:03.951008 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:06:03.951029 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:06:03.951497 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:06:03.951981 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:06:03.951984 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:06:03.951986 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:06:03.952170 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:06:03.952824 (+ 654us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:06:03.952830 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:06:03.953177 (+ 347us) client_negotiation.cc:769] Sending connection context
0811 12:06:03.953482 (+ 305us) client_negotiation.cc:240] Negotiation successful
0811 12:06:03.954030 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":342,"threads_started":1}
I20260811 12:06:03.956926 31936 heartbeater.cc:347] Connected to a master server at 127.9.221.190:32893
I20260811 12:06:03.957239 31936 heartbeater.cc:464] Registering TS with master...
I20260811 12:06:03.958012 31936 heartbeater.cc:511] Master 127.9.221.190:32893 requested a full tablet report, sending...
I20260811 12:06:03.960163 31665 ts_manager.cc:194] Registered new tserver with Master: 380d1e238e15456d98dee6b99cdf4f78 (127.9.221.129:43519)
I20260811 12:06:03.961697 31665 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.129@KRBTEST.COM'} at 127.9.221.129:38911
WARNING: no policy specified for kudu/127.9.221.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab.
Entry for principal kudu/127.9.221.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab.
I20260811 12:06:03.997097 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.130:0
--local_ip_for_outbound_sockets=127.9.221.130
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:32893
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab
--principal=kudu/127.9.221.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:37169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:06:04.337845 31946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:06:04.338148 31946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:06:04.338323 31946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:06:04.345265 31946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:06:04.345403 31946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:06:04.345602 31946 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.130
I20260811 12:06:04.355221 31946 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:37169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab
--principal=kudu/127.9.221.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:32893
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.130
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:06:04.356428 31946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:06:04.357673 31946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:06:04.370914 31953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:06:04.371057 31954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:06:04.371059 31957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:06:04.372260 31946 server_base.cc:1034] running on GCE node
I20260811 12:06:04.373070 31946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:06:04.374166 31946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:06:04.375377 31946 hybrid_clock.cc:648] HybridClock initialized: now 1786449964375327 us; error 45 us; skew 500 ppm
I20260811 12:06:04.375761 31946 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:06:04 dist-test-slave-mjjr krb5kdc[31590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449964, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:06:04.379156 31946 init.cc:377] Logged in from keytab as kudu/127.9.221.130@KRBTEST.COM (short username kudu)
I20260811 12:06:04.380656 31946 webserver.cc:460] Webserver started at http://127.9.221.130:41937/ using document root <none> and password file <none>
I20260811 12:06:04.381372 31946 fs_manager.cc:359] Metadata directory not provided
I20260811 12:06:04.381533 31946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:06:04.381862 31946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:06:04.385149 31946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-1/data/instance:
uuid: "43d08b9a29cf42a48adb34153409186a"
format_stamp: "Formatted at 2026-08-11 12:06:04 on dist-test-slave-mjjr"
server_key: "f0f3ff1b9439b238d8bea92bddd09620"
server_key_iv: "ecca2f464b5cf5fc19d63d5ef807c261"
server_key_version: "encryptionkey@0"
I20260811 12:06:04.385955 31946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance:
uuid: "43d08b9a29cf42a48adb34153409186a"
format_stamp: "Formatted at 2026-08-11 12:06:04 on dist-test-slave-mjjr"
server_key: "f0f3ff1b9439b238d8bea92bddd09620"
server_key_iv: "ecca2f464b5cf5fc19d63d5ef807c261"
server_key_version: "encryptionkey@0"
I20260811 12:06:04.391096 31946 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 12:06:04.394565 31965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:06:04.395872 31946 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:06:04.396046 31946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-1/wal
uuid: "43d08b9a29cf42a48adb34153409186a"
format_stamp: "Formatted at 2026-08-11 12:06:04 on dist-test-slave-mjjr"
server_key: "f0f3ff1b9439b238d8bea92bddd09620"
server_key_iv: "ecca2f464b5cf5fc19d63d5ef807c261"
server_key_version: "encryptionkey@0"
I20260811 12:06:04.396241 31946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:06:04.410382 31946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:06:04.414532 31946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:06:04.414763 31946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:06:04.415570 31946 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:06:04.416826 31946 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:06:04.416898 31946 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:06:04.416990 31946 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:06:04.417035 31946 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:06:04.448143 31946 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.130:44905
I20260811 12:06:04.448187 32122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.130:44905 every 8 connection(s)
I20260811 12:06:04.449368 31946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
I20260811 12:06:04.454250 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 31946
I20260811 12:06:04.454473 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance
I20260811 12:06:04.455057 10102 external_mini_cluster.cc:1428] Setting key dad9d531be139812f2948301f7fabc0a
Aug 11 12:06:04 dist-test-slave-mjjr krb5kdc[31590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449964, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.130@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:06:04.471014 31939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:04.451804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:54163 (local address 127.9.221.190:32893)
0811 12:06:04.451993 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:04.451997 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:04.453083 (+ 1086us) server_negotiation.cc:407] Connection header received
0811 12:06:04.455354 (+ 2271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:04.455358 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:04.455430 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:04.455516 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:06:04.457302 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:04.458045 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:04.460580 (+ 2535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:04.461001 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:04.461093 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:04.465185 (+ 4092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:06:04.465190 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:06:04.465193 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:06:04.465223 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:06:04.467943 (+ 2720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:04.468715 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:04.468719 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:04.468722 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:04.468845 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:04.469284 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:04.469288 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:04.469290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:04.469564 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:06:04.469720 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:04.470403 (+ 683us) server_negotiation.cc:299] Negotiation successful
0811 12:06:04.470633 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 12:06:04.471951 32126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:04.452212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.130:54163)
0811 12:06:04.452898 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:04.452940 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 12:06:04.455065 (+ 2125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:06:04.455824 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:06:04.455832 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:06:04.456444 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:06:04.457125 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:04.457153 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:04.458215 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:04.458223 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:04.460432 (+ 2209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:04.460451 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:04.461225 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:04.461234 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:04.461555 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:04.462235 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:06:04.462266 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:06:04.464965 (+ 2699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:06:04.468076 (+ 3111us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:06:04.468084 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:06:04.468097 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:06:04.468581 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:06:04.468955 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:06:04.468962 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:06:04.468966 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:06:04.469156 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:06:04.469692 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:06:04.469699 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:06:04.470007 (+ 308us) client_negotiation.cc:769] Sending connection context
0811 12:06:04.470316 (+ 309us) client_negotiation.cc:240] Negotiation successful
0811 12:06:04.470814 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":270,"threads_started":1}
I20260811 12:06:04.473418 32124 heartbeater.cc:347] Connected to a master server at 127.9.221.190:32893
I20260811 12:06:04.473706 32124 heartbeater.cc:464] Registering TS with master...
I20260811 12:06:04.474422 32124 heartbeater.cc:511] Master 127.9.221.190:32893 requested a full tablet report, sending...
I20260811 12:06:04.475929 31665 ts_manager.cc:194] Registered new tserver with Master: 43d08b9a29cf42a48adb34153409186a (127.9.221.130:44905)
I20260811 12:06:04.477209 31665 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.130@KRBTEST.COM'} at 127.9.221.130:54163
WARNING: no policy specified for kudu/127.9.221.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab.
Entry for principal kudu/127.9.221.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab.
I20260811 12:06:04.516788 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.131:0
--local_ip_for_outbound_sockets=127.9.221.131
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:32893
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab
--principal=kudu/127.9.221.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:37169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:06:04.849882 32132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:06:04.850147 32132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:06:04.850252 32132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:06:04.857600 32132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:06:04.857704 32132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:06:04.857836 32132 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.131
I20260811 12:06:04.867471 32132 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:37169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab
--principal=kudu/127.9.221.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:32893
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.131
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:06:04.868798 32132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:06:04.870215 32132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:06:04.883399 32140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:06:04.883430 32143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:06:04.883592 32141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:06:04.884181 32132 server_base.cc:1034] running on GCE node
I20260811 12:06:04.884779 32132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:06:04.885649 32132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:06:04.886880 32132 hybrid_clock.cc:648] HybridClock initialized: now 1786449964886829 us; error 39 us; skew 500 ppm
I20260811 12:06:04.887192 32132 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:06:04 dist-test-slave-mjjr krb5kdc[31590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449964, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:06:04.890239 32132 init.cc:377] Logged in from keytab as kudu/127.9.221.131@KRBTEST.COM (short username kudu)
I20260811 12:06:04.891685 32132 webserver.cc:460] Webserver started at http://127.9.221.131:37811/ using document root <none> and password file <none>
I20260811 12:06:04.892364 32132 fs_manager.cc:359] Metadata directory not provided
I20260811 12:06:04.892522 32132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:06:04.892813 32132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:06:04.895763 32132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-2/data/instance:
uuid: "d7bf3363e68843e189b30a08dde3b52d"
format_stamp: "Formatted at 2026-08-11 12:06:04 on dist-test-slave-mjjr"
server_key: "5a0fef63527135fba81646238b899a8a"
server_key_iv: "4116aa837c816cac98532681b80e9e35"
server_key_version: "encryptionkey@0"
I20260811 12:06:04.896631 32132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance:
uuid: "d7bf3363e68843e189b30a08dde3b52d"
format_stamp: "Formatted at 2026-08-11 12:06:04 on dist-test-slave-mjjr"
server_key: "5a0fef63527135fba81646238b899a8a"
server_key_iv: "4116aa837c816cac98532681b80e9e35"
server_key_version: "encryptionkey@0"
I20260811 12:06:04.901744 32132 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260811 12:06:04.905189 32151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:06:04.906445 32132 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 12:06:04.906651 32132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-2/wal
uuid: "d7bf3363e68843e189b30a08dde3b52d"
format_stamp: "Formatted at 2026-08-11 12:06:04 on dist-test-slave-mjjr"
server_key: "5a0fef63527135fba81646238b899a8a"
server_key_iv: "4116aa837c816cac98532681b80e9e35"
server_key_version: "encryptionkey@0"
I20260811 12:06:04.906859 32132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:06:04.923378 32132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:06:04.927191 32132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:06:04.927488 32132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:06:04.928506 32132 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:06:04.929885 32132 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:06:04.930004 32132 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:06:04.930166 32132 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:06:04.930238 32132 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:06:04.964293 32132 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.131:38705
I20260811 12:06:04.964337 32317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.131:38705 every 8 connection(s)
I20260811 12:06:04.964821 31936 heartbeater.cc:503] Master 127.9.221.190:32893 was elected leader, sending a full tablet report...
I20260811 12:06:04.965904 32132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
I20260811 12:06:04.971117 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 32132
I20260811 12:06:04.971320 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance
I20260811 12:06:04.971868 10102 external_mini_cluster.cc:1428] Setting key 7025c549785b1fd1823c6c09a1a3b0a0
Aug 11 12:06:04 dist-test-slave-mjjr krb5kdc[31590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449964, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.131@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:06:04.985452 31939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:04.968564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:54321 (local address 127.9.221.190:32893)
0811 12:06:04.968811 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:04.968818 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:04.969976 (+ 1158us) server_negotiation.cc:407] Connection header received
0811 12:06:04.971940 (+ 1964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:04.971946 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:04.972032 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:04.972147 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:06:04.974078 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:04.975016 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:04.976630 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:04.976997 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:04.977087 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:04.980320 (+ 3233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:06:04.980324 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:06:04.980328 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:06:04.980357 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:06:04.982761 (+ 2404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:04.983397 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:04.983401 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:04.983403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:04.983526 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:04.983989 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:04.983993 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:04.983995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:04.984236 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:06:04.984345 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:04.984990 (+ 645us) server_negotiation.cc:299] Negotiation successful
0811 12:06:04.985155 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 12:06:04.986358 32321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:04.969088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.131:54321)
0811 12:06:04.969806 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:04.969846 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 12:06:04.971623 (+ 1777us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:06:04.972317 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:06:04.972325 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:06:04.973042 (+ 717us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:06:04.973894 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:04.973922 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:04.975197 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:04.975201 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:04.976496 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:04.976508 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:04.977182 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:04.977187 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:04.977371 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:04.977973 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:06:04.978000 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:06:04.980139 (+ 2139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:06:04.982883 (+ 2744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:06:04.982889 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:06:04.982899 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:06:04.983290 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:06:04.983635 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:06:04.983638 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:06:04.983642 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:06:04.983866 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:06:04.984343 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:06:04.984349 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:06:04.984654 (+ 305us) client_negotiation.cc:769] Sending connection context
0811 12:06:04.984875 (+ 221us) client_negotiation.cc:240] Negotiation successful
0811 12:06:04.985229 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":260,"threads_started":1}
I20260811 12:06:04.987892 32318 heartbeater.cc:347] Connected to a master server at 127.9.221.190:32893
I20260811 12:06:04.988152 32318 heartbeater.cc:464] Registering TS with master...
I20260811 12:06:04.988780 32318 heartbeater.cc:511] Master 127.9.221.190:32893 requested a full tablet report, sending...
I20260811 12:06:04.990087 31665 ts_manager.cc:194] Registered new tserver with Master: d7bf3363e68843e189b30a08dde3b52d (127.9.221.131:38705)
I20260811 12:06:04.990936 31665 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.131@KRBTEST.COM'} at 127.9.221.131:54321
I20260811 12:06:04.999239 10102 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:06:05.015797 31939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:05.003708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41540 (local address 127.9.221.190:32893)
0811 12:06:05.003871 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:05.003875 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:05.004388 (+ 513us) server_negotiation.cc:407] Connection header received
0811 12:06:05.004515 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:05.004519 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:05.004573 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:05.004650 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:06:05.006032 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:05.006743 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:05.008054 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:05.008431 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:05.008520 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:05.011047 (+ 2527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:06:05.011052 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:06:05.011055 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:06:05.011120 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:06:05.013493 (+ 2373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:05.014117 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:05.014120 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:05.014122 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:05.014244 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:05.014551 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:05.014554 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:05.014556 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:05.014845 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:06:05.015028 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:05.015278 (+ 250us) server_negotiation.cc:299] Negotiation successful
0811 12:06:05.015422 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 12:06:05.019466 31665 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41540:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:06:05.026310 31665 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:06:05.050885 32338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:05.037750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:43519 (local address 127.0.0.1:39368)
0811 12:06:05.038542 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:05.038599 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 12:06:05.038824 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:06:05.040283 (+ 1459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:06:05.040291 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:06:05.040332 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:06:05.040886 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:05.040902 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:05.043692 (+ 2790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:05.043701 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:05.046847 (+ 3146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:05.046868 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:05.049763 (+ 2895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:05.049771 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:05.049997 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:05.050056 (+ 59us) client_negotiation.cc:769] Sending connection context
0811 12:06:05.050166 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 12:06:05.050398 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":329,"threads_started":1}
I20260811 12:06:05.053203 32339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:05.038049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39368 (local address 127.9.221.129:43519)
0811 12:06:05.039551 (+ 1502us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:05.039563 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:05.039599 (+ 36us) server_negotiation.cc:407] Connection header received
0811 12:06:05.039747 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:05.039757 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:05.040030 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:05.041138 (+ 1108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:06:05.041483 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:05.043448 (+ 1965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:05.047117 (+ 3669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:05.049467 (+ 2350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:05.052150 (+ 2683us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:05.052540 (+ 390us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:05.052624 (+ 84us) server_negotiation.cc:299] Negotiation successful
0811 12:06:05.052844 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1295,"thread_start_us":1125,"threads_started":1}
I20260811 12:06:05.057844 31844 tablet_service.cc:1467] Processing CreateTablet for tablet 8738758dc57d46abac8fb00fd671b41f (DEFAULT_TABLE table=test-table [id=5d35a32284dc48b89063f9a4696d3b64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:06:05.059243 31844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8738758dc57d46abac8fb00fd671b41f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:06:05.064026 32338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:05.038886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.130:44905 (local address 127.0.0.1:49918)
0811 12:06:05.051184 (+ 12298us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:05.051206 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:06:05.051320 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:06:05.057352 (+ 6032us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:06:05.057357 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:06:05.057379 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:06:05.057589 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:05.057601 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:05.059621 (+ 2020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:05.059627 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:05.061631 (+ 2004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:05.061648 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:05.063100 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:05.063105 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:05.063276 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:05.063389 (+ 113us) client_negotiation.cc:769] Sending connection context
0811 12:06:05.063490 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 12:06:05.063644 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12177,"mutex_wait_us":30,"thread_start_us":10902,"threads_started":1}
I20260811 12:06:05.064101 32347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:05.039328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49918 (local address 127.9.221.130:44905)
0811 12:06:05.055627 (+ 16299us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:05.055852 (+ 225us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:05.055892 (+ 40us) server_negotiation.cc:407] Connection header received
0811 12:06:05.056020 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:05.056031 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:05.056309 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:05.057771 (+ 1462us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:06:05.057987 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:05.059467 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:05.061818 (+ 2351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:05.062926 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:05.063076 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:05.063260 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:05.063573 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 12:06:05.063767 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16064,"thread_start_us":16308,"threads_started":1}
I20260811 12:06:05.065207 32345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:05.038680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.131:38705 (local address 127.0.0.1:59018)
0811 12:06:05.050047 (+ 11367us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:05.050065 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:06:05.050181 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:06:05.057757 (+ 7576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:06:05.057762 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:06:05.057785 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:06:05.058168 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:05.058180 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:05.060087 (+ 1907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:05.060095 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:05.062015 (+ 1920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:05.062029 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:05.064364 (+ 2335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:05.064372 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:05.064560 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:05.064592 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 12:06:05.064689 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 12:06:05.064814 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11233,"mutex_wait_us":2,"thread_start_us":18734,"threads_started":1}
I20260811 12:06:05.065436 32346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:05.039256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59018 (local address 127.9.221.131:38705)
0811 12:06:05.057044 (+ 17788us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:05.057053 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:05.057090 (+ 37us) server_negotiation.cc:407] Connection header received
0811 12:06:05.057193 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:05.057199 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:05.057420 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:05.057636 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:06:05.058372 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:05.059960 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:05.062231 (+ 2271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:05.063709 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:05.063830 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:05.064045 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:05.064761 (+ 716us) server_negotiation.cc:299] Negotiation successful
0811 12:06:05.064914 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17613,"thread_start_us":17457,"threads_started":1}
I20260811 12:06:05.067997 32026 tablet_service.cc:1467] Processing CreateTablet for tablet 8738758dc57d46abac8fb00fd671b41f (DEFAULT_TABLE table=test-table [id=5d35a32284dc48b89063f9a4696d3b64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:06:05.069217 32026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8738758dc57d46abac8fb00fd671b41f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:06:05.068881 32218 tablet_service.cc:1467] Processing CreateTablet for tablet 8738758dc57d46abac8fb00fd671b41f (DEFAULT_TABLE table=test-table [id=5d35a32284dc48b89063f9a4696d3b64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:06:05.070226 32218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8738758dc57d46abac8fb00fd671b41f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:06:05.078572 32348 tablet_bootstrap.cc:492] T 8738758dc57d46abac8fb00fd671b41f P 380d1e238e15456d98dee6b99cdf4f78: Bootstrap starting.
I20260811 12:06:05.081801 32349 tablet_bootstrap.cc:492] T 8738758dc57d46abac8fb00fd671b41f P 43d08b9a29cf42a48adb34153409186a: Bootstrap starting.
I20260811 12:06:05.082809 32348 tablet_bootstrap.cc:654] T 8738758dc57d46abac8fb00fd671b41f P 380d1e238e15456d98dee6b99cdf4f78: Neither blocks nor log segments found. Creating new log.
I20260811 12:06:05.083926 32348 log.cc:824] T 8738758dc57d46abac8fb00fd671b41f P 380d1e238e15456d98dee6b99cdf4f78: Log is configured to *not* fsync() on all Append() calls
I20260811 12:06:05.085256 32349 tablet_bootstrap.cc:654] T 8738758dc57d46abac8fb00fd671b41f P 43d08b9a29cf42a48adb34153409186a: Neither blocks nor log segments found. Creating new log.
I20260811 12:06:05.086287 32354 tablet_bootstrap.cc:492] T 8738758dc57d46abac8fb00fd671b41f P d7bf3363e68843e189b30a08dde3b52d: Bootstrap starting.
I20260811 12:06:05.086573 32349 log.cc:824] T 8738758dc57d46abac8fb00fd671b41f P 43d08b9a29cf42a48adb34153409186a: Log is configured to *not* fsync() on all Append() calls
I20260811 12:06:05.089599 32349 tablet_bootstrap.cc:492] T 8738758dc57d46abac8fb00fd671b41f P 43d08b9a29cf42a48adb34153409186a: No bootstrap required, opened a new log
I20260811 12:06:05.089605 32348 tablet_bootstrap.cc:492] T 8738758dc57d46abac8fb00fd671b41f P 380d1e238e15456d98dee6b99cdf4f78: No bootstrap required, opened a new log
I20260811 12:06:05.089874 32349 ts_tablet_manager.cc:1397] T 8738758dc57d46abac8fb00fd671b41f P 43d08b9a29cf42a48adb34153409186a: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 12:06:05.089874 32348 ts_tablet_manager.cc:1397] T 8738758dc57d46abac8fb00fd671b41f P 380d1e238e15456d98dee6b99cdf4f78: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260811 12:06:05.089985 32354 tablet_bootstrap.cc:654] T 8738758dc57d46abac8fb00fd671b41f P d7bf3363e68843e189b30a08dde3b52d: Neither blocks nor log segments found. Creating new log.
I20260811 12:06:05.090917 32354 log.cc:824] T 8738758dc57d46abac8fb00fd671b41f P d7bf3363e68843e189b30a08dde3b52d: Log is configured to *not* fsync() on all Append() calls
I20260811 12:06:05.093314 32354 tablet_bootstrap.cc:492] T 8738758dc57d46abac8fb00fd671b41f P d7bf3363e68843e189b30a08dde3b52d: No bootstrap required, opened a new log
I20260811 12:06:05.093577 32354 ts_tablet_manager.cc:1397] T 8738758dc57d46abac8fb00fd671b41f P d7bf3363e68843e189b30a08dde3b52d: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 12:06:05.100613 32348 raft_consensus.cc:348] T 8738758dc57d46abac8fb00fd671b41f P 380d1e238e15456d98dee6b99cdf4f78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "380d1e238e15456d98dee6b99cdf4f78" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43519 } } peers { permanent_uuid: "43d08b9a29cf42a48adb34153409186a" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 44905 } } peers { permanent_uuid: "d7bf3363e68843e189b30a08dde3b52d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 38705 } }
I20260811 12:06:05.100630 32349 raft_consensus.cc:348] T 8738758dc57d46abac8fb00fd671b41f P 43d08b9a29cf42a48adb34153409186a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "380d1e238e15456d98dee6b99cdf4f78" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43519 } } peers { permanent_uuid: "43d08b9a29cf42a48adb34153409186a" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 44905 } } peers { permanent_uuid: "d7bf3363e68843e189b30a08dde3b52d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 38705 } }
I20260811 12:06:05.101054 32348 raft_consensus.cc:374] T 8738758dc57d46abac8fb00fd671b41f P 380d1e238e15456d98dee6b99cdf4f78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:06:05.101050 32349 raft_consensus.cc:374] T 8738758dc57d46abac8fb00fd671b41f P 43d08b9a29cf42a48adb34153409186a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:06:05.101153 32349 raft_consensus.cc:729] T 8738758dc57d46abac8fb00fd671b41f P 43d08b9a29cf42a48adb34153409186a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43d08b9a29cf42a48adb34153409186a, State: Initialized, Role: FOLLOWER
I20260811 12:06:05.101152 32348 raft_consensus.cc:729] T 8738758dc57d46abac8fb00fd671b41f P 380d1e238e15456d98dee6b99cdf4f78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 380d1e238e15456d98dee6b99cdf4f78, State: Initialized, Role: FOLLOWER
I20260811 12:06:05.101042 32354 raft_consensus.cc:348] T 8738758dc57d46abac8fb00fd671b41f P d7bf3363e68843e189b30a08dde3b52d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "380d1e238e15456d98dee6b99cdf4f78" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43519 } } peers { permanent_uuid: "43d08b9a29cf42a48adb34153409186a" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 44905 } } peers { permanent_uuid: "d7bf3363e68843e189b30a08dde3b52d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 38705 } }
I20260811 12:06:05.101475 32354 raft_consensus.cc:374] T 8738758dc57d46abac8fb00fd671b41f P d7bf3363e68843e189b30a08dde3b52d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:06:05.101588 32354 raft_consensus.cc:729] T 8738758dc57d46abac8fb00fd671b41f P d7bf3363e68843e189b30a08dde3b52d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7bf3363e68843e189b30a08dde3b52d, State: Initialized, Role: FOLLOWER
I20260811 12:06:05.101774 32348 consensus_queue.cc:260] T 8738758dc57d46abac8fb00fd671b41f P 380d1e238e15456d98dee6b99cdf4f78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "380d1e238e15456d98dee6b99cdf4f78" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43519 } } peers { permanent_uuid: "43d08b9a29cf42a48adb34153409186a" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 44905 } } peers { permanent_uuid: "d7bf3363e68843e189b30a08dde3b52d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 38705 } }
I20260811 12:06:05.101774 32349 consensus_queue.cc:260] T 8738758dc57d46abac8fb00fd671b41f P 43d08b9a29cf42a48adb34153409186a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "380d1e238e15456d98dee6b99cdf4f78" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43519 } } peers { permanent_uuid: "43d08b9a29cf42a48adb34153409186a" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 44905 } } peers { permanent_uuid: "d7bf3363e68843e189b30a08dde3b52d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 38705 } }
I20260811 12:06:05.102276 32354 consensus_queue.cc:260] T 8738758dc57d46abac8fb00fd671b41f P d7bf3363e68843e189b30a08dde3b52d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "380d1e238e15456d98dee6b99cdf4f78" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43519 } } peers { permanent_uuid: "43d08b9a29cf42a48adb34153409186a" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 44905 } } peers { permanent_uuid: "d7bf3363e68843e189b30a08dde3b52d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 38705 } }
I20260811 12:06:05.103766 32124 heartbeater.cc:503] Master 127.9.221.190:32893 was elected leader, sending a full tablet report...
I20260811 12:06:05.103861 32349 ts_tablet_manager.cc:1428] T 8738758dc57d46abac8fb00fd671b41f P 43d08b9a29cf42a48adb34153409186a: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260811 12:06:05.105309 32348 ts_tablet_manager.cc:1428] T 8738758dc57d46abac8fb00fd671b41f P 380d1e238e15456d98dee6b99cdf4f78: Time spent starting tablet: real 0.015s user 0.010s sys 0.005s
I20260811 12:06:05.109259 32318 heartbeater.cc:503] Master 127.9.221.190:32893 was elected leader, sending a full tablet report...
I20260811 12:06:05.109807 32354 ts_tablet_manager.cc:1428] T 8738758dc57d46abac8fb00fd671b41f P d7bf3363e68843e189b30a08dde3b52d: Time spent starting tablet: real 0.016s user 0.008s sys 0.009s
W20260811 12:06:05.181875 31937 tablet.cc:2369] T 8738758dc57d46abac8fb00fd671b41f P 380d1e238e15456d98dee6b99cdf4f78: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:06:05.188673 32361 raft_consensus.cc:482] T 8738758dc57d46abac8fb00fd671b41f P 43d08b9a29cf42a48adb34153409186a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:06:05.188987 32361 raft_consensus.cc:504] T 8738758dc57d46abac8fb00fd671b41f P 43d08b9a29cf42a48adb34153409186a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "380d1e238e15456d98dee6b99cdf4f78" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43519 } } peers { permanent_uuid: "43d08b9a29cf42a48adb34153409186a" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 44905 } } peers { permanent_uuid: "d7bf3363e68843e189b30a08dde3b52d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 38705 } }
I20260811 12:06:05.190665 32361 leader_election.cc:288] T 8738758dc57d46abac8fb00fd671b41f P 43d08b9a29cf42a48adb34153409186a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 380d1e238e15456d98dee6b99cdf4f78 (127.9.221.129:43519), d7bf3363e68843e189b30a08dde3b52d (127.9.221.131:38705)
I20260811 12:06:05.197582 32339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:05.191198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:39135 (local address 127.9.221.129:43519)
0811 12:06:05.191449 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:05.191458 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:05.192197 (+ 739us) server_negotiation.cc:407] Connection header received
0811 12:06:05.192436 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:05.192440 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:05.192498 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:05.192600 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:06:05.193217 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:05.193943 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:05.195873 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:05.196627 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:05.196800 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:05.196886 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:05.197201 (+ 315us) server_negotiation.cc:299] Negotiation successful
0811 12:06:05.197321 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 12:06:05.197639 32346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:05.191311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:47535 (local address 127.9.221.131:38705)
0811 12:06:05.191575 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:05.191584 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:05.192274 (+ 690us) server_negotiation.cc:407] Connection header received
0811 12:06:05.192461 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:05.192466 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:05.192524 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:05.192622 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:06:05.193284 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:05.194017 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:05.195941 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:05.196702 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:05.196885 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:05.196973 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:05.197254 (+ 281us) server_negotiation.cc:299] Negotiation successful
0811 12:06:05.197375 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 12:06:05.197758 32365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:05.190948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:43519 (local address 127.9.221.130:39135)
0811 12:06:05.192084 (+ 1136us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:05.192104 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:06:05.192271 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:06:05.192614 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:06:05.192618 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:06:05.192639 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:06:05.193066 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:05.193078 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:05.194072 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:05.194080 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:05.195720 (+ 1640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:05.195732 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:05.196871 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:05.196877 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:05.197042 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:05.197068 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 12:06:05.197174 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 12:06:05.197296 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1012,"mutex_wait_us":38,"thread_start_us":759,"threads_started":1}
I20260811 12:06:05.197798 32366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:05.190971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.131:38705 (local address 127.9.221.130:47535)
0811 12:06:05.192179 (+ 1208us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:05.192193 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 12:06:05.192301 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:06:05.192636 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:06:05.192641 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:06:05.192662 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:06:05.193111 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:05.193120 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:05.194161 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:05.194166 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:05.195790 (+ 1624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:05.195803 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:05.196922 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:05.196928 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:05.197102 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:05.197124 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 12:06:05.197229 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 12:06:05.197348 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1100,"mutex_wait_us":40,"thread_start_us":888,"threads_started":1}
I20260811 12:06:05.198792 31871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8738758dc57d46abac8fb00fd671b41f" candidate_uuid: "43d08b9a29cf42a48adb34153409186a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "380d1e238e15456d98dee6b99cdf4f78" is_pre_election: true
I20260811 12:06:05.198853 32251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8738758dc57d46abac8fb00fd671b41f" candidate_uuid: "43d08b9a29cf42a48adb34153409186a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7bf3363e68843e189b30a08dde3b52d" is_pre_election: true
I20260811 12:06:05.199355 31871 raft_consensus.cc:2454] T 8738758dc57d46abac8fb00fd671b41f P 380d1e238e15456d98dee6b99cdf4f78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43d08b9a29cf42a48adb34153409186a in term 0.
I20260811 12:06:05.199375 32251 raft_consensus.cc:2454] T 8738758dc57d46abac8fb00fd671b41f P d7bf3363e68843e189b30a08dde3b52d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43d08b9a29cf42a48adb34153409186a in term 0.
I20260811 12:06:05.200114 31969 leader_election.cc:302] T 8738758dc57d46abac8fb00fd671b41f P 43d08b9a29cf42a48adb34153409186a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 380d1e238e15456d98dee6b99cdf4f78, 43d08b9a29cf42a48adb34153409186a; no voters:
I20260811 12:06:05.200536 32361 raft_consensus.cc:2781] T 8738758dc57d46abac8fb00fd671b41f P 43d08b9a29cf42a48adb34153409186a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:06:05.200687 32361 raft_consensus.cc:482] T 8738758dc57d46abac8fb00fd671b41f P 43d08b9a29cf42a48adb34153409186a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:06:05.200798 32361 raft_consensus.cc:3037] T 8738758dc57d46abac8fb00fd671b41f P 43d08b9a29cf42a48adb34153409186a [term 0 FOLLOWER]: Advancing to term 1
W20260811 12:06:05.201267 32125 tablet.cc:2369] T 8738758dc57d46abac8fb00fd671b41f P 43d08b9a29cf42a48adb34153409186a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:06:05.203647 32361 raft_consensus.cc:504] T 8738758dc57d46abac8fb00fd671b41f P 43d08b9a29cf42a48adb34153409186a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "380d1e238e15456d98dee6b99cdf4f78" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43519 } } peers { permanent_uuid: "43d08b9a29cf42a48adb34153409186a" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 44905 } } peers { permanent_uuid: "d7bf3363e68843e189b30a08dde3b52d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 38705 } }
I20260811 12:06:05.204319 32361 leader_election.cc:288] T 8738758dc57d46abac8fb00fd671b41f P 43d08b9a29cf42a48adb34153409186a [CANDIDATE]: Term 1 election: Requested vote from peers 380d1e238e15456d98dee6b99cdf4f78 (127.9.221.129:43519), d7bf3363e68843e189b30a08dde3b52d (127.9.221.131:38705)
I20260811 12:06:05.204663 31871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8738758dc57d46abac8fb00fd671b41f" candidate_uuid: "43d08b9a29cf42a48adb34153409186a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "380d1e238e15456d98dee6b99cdf4f78"
I20260811 12:06:05.204881 31871 raft_consensus.cc:3037] T 8738758dc57d46abac8fb00fd671b41f P 380d1e238e15456d98dee6b99cdf4f78 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:06:05.204865 32251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8738758dc57d46abac8fb00fd671b41f" candidate_uuid: "43d08b9a29cf42a48adb34153409186a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7bf3363e68843e189b30a08dde3b52d"
I20260811 12:06:05.205068 32251 raft_consensus.cc:3037] T 8738758dc57d46abac8fb00fd671b41f P d7bf3363e68843e189b30a08dde3b52d [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:06:05.207059 31871 raft_consensus.cc:2454] T 8738758dc57d46abac8fb00fd671b41f P 380d1e238e15456d98dee6b99cdf4f78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43d08b9a29cf42a48adb34153409186a in term 1.
I20260811 12:06:05.207307 32251 raft_consensus.cc:2454] T 8738758dc57d46abac8fb00fd671b41f P d7bf3363e68843e189b30a08dde3b52d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43d08b9a29cf42a48adb34153409186a in term 1.
I20260811 12:06:05.207434 31969 leader_election.cc:302] T 8738758dc57d46abac8fb00fd671b41f P 43d08b9a29cf42a48adb34153409186a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 380d1e238e15456d98dee6b99cdf4f78, 43d08b9a29cf42a48adb34153409186a; no voters:
I20260811 12:06:05.207736 32361 raft_consensus.cc:2781] T 8738758dc57d46abac8fb00fd671b41f P 43d08b9a29cf42a48adb34153409186a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:06:05.208222 32361 raft_consensus.cc:686] T 8738758dc57d46abac8fb00fd671b41f P 43d08b9a29cf42a48adb34153409186a [term 1 LEADER]: Becoming Leader. State: Replica: 43d08b9a29cf42a48adb34153409186a, State: Running, Role: LEADER
I20260811 12:06:05.208658 32361 consensus_queue.cc:237] T 8738758dc57d46abac8fb00fd671b41f P 43d08b9a29cf42a48adb34153409186a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "380d1e238e15456d98dee6b99cdf4f78" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43519 } } peers { permanent_uuid: "43d08b9a29cf42a48adb34153409186a" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 44905 } } peers { permanent_uuid: "d7bf3363e68843e189b30a08dde3b52d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 38705 } }
I20260811 12:06:05.213060 31665 catalog_manager.cc:5495] T 8738758dc57d46abac8fb00fd671b41f P 43d08b9a29cf42a48adb34153409186a reported cstate change: term changed from 0 to 1, leader changed from <none> to 43d08b9a29cf42a48adb34153409186a (127.9.221.130). New cstate: current_term: 1 leader_uuid: "43d08b9a29cf42a48adb34153409186a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "380d1e238e15456d98dee6b99cdf4f78" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43d08b9a29cf42a48adb34153409186a" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 44905 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7bf3363e68843e189b30a08dde3b52d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 38705 } health_report { overall_health: UNKNOWN } } }
W20260811 12:06:05.217939 32319 tablet.cc:2369] T 8738758dc57d46abac8fb00fd671b41f P d7bf3363e68843e189b30a08dde3b52d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:06:05.252355 32347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:05.247435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49934 (local address 127.9.221.130:44905)
0811 12:06:05.247634 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:05.247672 (+ 38us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:05.247697 (+ 25us) server_negotiation.cc:407] Connection header received
0811 12:06:05.247787 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:05.247791 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:05.247867 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:05.248004 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:06:05.248340 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:05.249044 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:05.250326 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:05.250708 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:05.250795 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:05.251242 (+ 447us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:06:05.251808 (+ 566us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:06:05.251923 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:05.252028 (+ 105us) server_negotiation.cc:299] Negotiation successful
0811 12:06:05.252109 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 12:06:05.260393 31871 raft_consensus.cc:1261] T 8738758dc57d46abac8fb00fd671b41f P 380d1e238e15456d98dee6b99cdf4f78 [term 1 FOLLOWER]: Refusing update from remote peer 43d08b9a29cf42a48adb34153409186a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:06:05.260396 32251 raft_consensus.cc:1261] T 8738758dc57d46abac8fb00fd671b41f P d7bf3363e68843e189b30a08dde3b52d [term 1 FOLLOWER]: Refusing update from remote peer 43d08b9a29cf42a48adb34153409186a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:06:05.261374 32367 consensus_queue.cc:1035] T 8738758dc57d46abac8fb00fd671b41f P 43d08b9a29cf42a48adb34153409186a [LEADER]: Connected to new peer: Peer: permanent_uuid: "380d1e238e15456d98dee6b99cdf4f78" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:06:05.261719 32361 consensus_queue.cc:1035] T 8738758dc57d46abac8fb00fd671b41f P 43d08b9a29cf42a48adb34153409186a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7bf3363e68843e189b30a08dde3b52d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 38705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:06:05.273888 32377 mvcc.cc:205] Tried to move back new op lower bound from 7317299057696899072 to 7317299057498222592. Current Snapshot: MvccSnapshot[applied={T|T < 7317299057696899072}]
I20260811 12:06:05.289366 31665 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41540:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:06:05.289675 31665 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41540:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:06:05.293471 31665 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc: Sending DeleteTablet for 3 replicas of tablet 8738758dc57d46abac8fb00fd671b41f
I20260811 12:06:05.294646 32026 tablet_service.cc:1514] Processing DeleteTablet for tablet 8738758dc57d46abac8fb00fd671b41f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:06:05 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:49918
I20260811 12:06:05.294642 31844 tablet_service.cc:1514] Processing DeleteTablet for tablet 8738758dc57d46abac8fb00fd671b41f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:06:05 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:39368
I20260811 12:06:05.294804 32218 tablet_service.cc:1514] Processing DeleteTablet for tablet 8738758dc57d46abac8fb00fd671b41f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:06:05 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:59018
I20260811 12:06:05.295785 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 31755
I20260811 12:06:05.297015 32392 tablet_replica.cc:330] T 8738758dc57d46abac8fb00fd671b41f P d7bf3363e68843e189b30a08dde3b52d: stopping tablet replica
I20260811 12:06:05.297611 32392 raft_consensus.cc:2229] T 8738758dc57d46abac8fb00fd671b41f P d7bf3363e68843e189b30a08dde3b52d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:06:05.298353 32392 raft_consensus.cc:2258] T 8738758dc57d46abac8fb00fd671b41f P d7bf3363e68843e189b30a08dde3b52d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:06:05.299240 32390 tablet_replica.cc:330] T 8738758dc57d46abac8fb00fd671b41f P 43d08b9a29cf42a48adb34153409186a: stopping tablet replica
I20260811 12:06:05.299847 32391 tablet_replica.cc:330] T 8738758dc57d46abac8fb00fd671b41f P 380d1e238e15456d98dee6b99cdf4f78: stopping tablet replica
I20260811 12:06:05.300769 32391 raft_consensus.cc:2229] T 8738758dc57d46abac8fb00fd671b41f P 380d1e238e15456d98dee6b99cdf4f78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:06:05.300318 32390 raft_consensus.cc:2229] T 8738758dc57d46abac8fb00fd671b41f P 43d08b9a29cf42a48adb34153409186a [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:06:05.301208 32392 ts_tablet_manager.cc:1905] T 8738758dc57d46abac8fb00fd671b41f P d7bf3363e68843e189b30a08dde3b52d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:06:05.301783 32391 raft_consensus.cc:2258] T 8738758dc57d46abac8fb00fd671b41f P 380d1e238e15456d98dee6b99cdf4f78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:06:05.302297 32390 raft_consensus.cc:2258] T 8738758dc57d46abac8fb00fd671b41f P 43d08b9a29cf42a48adb34153409186a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:06:05.304792 32391 ts_tablet_manager.cc:1905] T 8738758dc57d46abac8fb00fd671b41f P 380d1e238e15456d98dee6b99cdf4f78: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:06:05.308946 32392 ts_tablet_manager.cc:1918] T 8738758dc57d46abac8fb00fd671b41f P d7bf3363e68843e189b30a08dde3b52d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:06:05.309132 32392 log.cc:1196] T 8738758dc57d46abac8fb00fd671b41f P d7bf3363e68843e189b30a08dde3b52d: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-2/wal/wals/8738758dc57d46abac8fb00fd671b41f
Aug 11 12:06:05 dist-test-slave-mjjr krb5kdc[31590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.129@KRBTEST.COM
I20260811 12:06:05.312631 32391 ts_tablet_manager.cc:1918] T 8738758dc57d46abac8fb00fd671b41f P 380d1e238e15456d98dee6b99cdf4f78: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:06:05.312777 32391 log.cc:1196] T 8738758dc57d46abac8fb00fd671b41f P 380d1e238e15456d98dee6b99cdf4f78: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-0/wal/wals/8738758dc57d46abac8fb00fd671b41f
I20260811 12:06:05.313265 32391 ts_tablet_manager.cc:1939] T 8738758dc57d46abac8fb00fd671b41f P 380d1e238e15456d98dee6b99cdf4f78: Deleting consensus metadata
I20260811 12:06:05.314996 31642 catalog_manager.cc:4841] TS 380d1e238e15456d98dee6b99cdf4f78 (127.9.221.129:43519): tablet 8738758dc57d46abac8fb00fd671b41f (table test-table [id=5d35a32284dc48b89063f9a4696d3b64]) successfully deleted
I20260811 12:06:05.315564 32390 ts_tablet_manager.cc:1905] T 8738758dc57d46abac8fb00fd671b41f P 43d08b9a29cf42a48adb34153409186a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:06:05.322000 32339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:05.297222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39384 (local address 127.9.221.129:43519)
0811 12:06:05.297443 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:05.297452 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:05.300179 (+ 2727us) server_negotiation.cc:407] Connection header received
0811 12:06:05.300480 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:05.300486 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:05.300565 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:05.300671 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:06:05.303059 (+ 2388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:05.304268 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:05.306748 (+ 2480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:05.307435 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:05.307537 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:05.313699 (+ 6162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:06:05.313715 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:06:05.313732 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:06:05.313773 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:06:05.318439 (+ 4666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:05.319322 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:05.319332 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:05.319336 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:05.319508 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:05.320030 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:05.320035 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:05.320039 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:05.320491 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:06:05.320751 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:05.321369 (+ 618us) server_negotiation.cc:299] Negotiation successful
0811 12:06:05.321619 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 12:06:05.322706 32390 ts_tablet_manager.cc:1918] T 8738758dc57d46abac8fb00fd671b41f P 43d08b9a29cf42a48adb34153409186a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:06:05.322866 32390 log.cc:1196] T 8738758dc57d46abac8fb00fd671b41f P 43d08b9a29cf42a48adb34153409186a: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-1/wal/wals/8738758dc57d46abac8fb00fd671b41f
I20260811 12:06:05.323438 32390 ts_tablet_manager.cc:1939] T 8738758dc57d46abac8fb00fd671b41f P 43d08b9a29cf42a48adb34153409186a: Deleting consensus metadata
I20260811 12:06:05.323719 32392 ts_tablet_manager.cc:1939] T 8738758dc57d46abac8fb00fd671b41f P d7bf3363e68843e189b30a08dde3b52d: Deleting consensus metadata
I20260811 12:06:05.323788 31929 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:06:05.325012 31642 catalog_manager.cc:4841] TS 43d08b9a29cf42a48adb34153409186a (127.9.221.130:44905): tablet 8738758dc57d46abac8fb00fd671b41f (table test-table [id=5d35a32284dc48b89063f9a4696d3b64]) successfully deleted
I20260811 12:06:05.325981 31641 catalog_manager.cc:4841] TS d7bf3363e68843e189b30a08dde3b52d (127.9.221.131:38705): tablet 8738758dc57d46abac8fb00fd671b41f (table test-table [id=5d35a32284dc48b89063f9a4696d3b64]) successfully deleted
I20260811 12:06:05.416051 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 31755
I20260811 12:06:05.439961 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 31946
Aug 11 12:06:05 dist-test-slave-mjjr krb5kdc[31590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.130@KRBTEST.COM
I20260811 12:06:05.465535 32347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:05.440791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49938 (local address 127.9.221.130:44905)
0811 12:06:05.440972 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:05.440977 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:05.441081 (+ 104us) server_negotiation.cc:407] Connection header received
0811 12:06:05.441321 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:05.441329 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:05.441415 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:05.441522 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:06:05.443270 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:05.444171 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:05.446235 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:05.446685 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:05.446787 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:05.457964 (+ 11177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:06:05.457981 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:06:05.457998 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:06:05.458048 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:06:05.462189 (+ 4141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:05.462984 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:05.462994 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:05.462998 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:05.463181 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:05.463740 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:05.463746 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:05.463750 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:05.464211 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:06:05.464553 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:05.464831 (+ 278us) server_negotiation.cc:299] Negotiation successful
0811 12:06:05.465090 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 12:06:05.466271 32116 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:06:05.583530 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 31946
I20260811 12:06:05.607632 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 32132
Aug 11 12:06:05 dist-test-slave-mjjr krb5kdc[31590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.131@KRBTEST.COM
I20260811 12:06:05.623124 32346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:05.608571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45246 (local address 127.9.221.131:38705)
0811 12:06:05.608804 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:05.608811 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:05.608831 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:06:05.609027 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:05.609039 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:05.609105 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:05.609216 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:06:05.610654 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:05.611370 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:05.612756 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:05.613134 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:05.613265 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:05.617601 (+ 4336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:06:05.617615 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:06:05.617630 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:06:05.617669 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:06:05.620590 (+ 2921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:05.621263 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:05.621269 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:05.621272 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:05.621412 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:05.621853 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:05.621856 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:05.621859 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:05.622207 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:06:05.622425 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:05.622687 (+ 262us) server_negotiation.cc:299] Negotiation successful
0811 12:06:05.622842 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 12:06:05.624007 32309 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:06:05.712677 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 32132
I20260811 12:06:05.736991 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 31608
I20260811 12:06:05.738355 31706 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:06:05.848616 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 31608
I20260811 12:06:05.883389 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:32893
--webserver_interface=127.9.221.190
--webserver_port=41771
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:37169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:32893
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:06:06.225689 32404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:06:06.226025 32404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:06:06.226186 32404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:06:06.233142 32404 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:06:06.233278 32404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:06:06.233381 32404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:06:06.233482 32404 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:06:06.233568 32404 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:06:06.243295 32404 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:37169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:32893
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:32893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.221.190
--webserver_port=41771
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:06:06.244635 32404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:06:06.245985 32404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:06:06.271327 32413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:06:06.271346 32414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:06:06.271452 32417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:06:06.272748 32404 server_base.cc:1034] running on GCE node
I20260811 12:06:06.273599 32404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:06:06.275223 32404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:06:06.276443 32404 hybrid_clock.cc:648] HybridClock initialized: now 1786449966276374 us; error 53 us; skew 500 ppm
I20260811 12:06:06.276829 32404 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:06:06 dist-test-slave-mjjr krb5kdc[31590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449966, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:06:06.285483 32404 init.cc:377] Logged in from keytab as kudu/127.9.221.190@KRBTEST.COM (short username kudu)
I20260811 12:06:06.287266 32404 webserver.cc:460] Webserver started at http://127.9.221.190:41771/ using document root <none> and password file <none>
I20260811 12:06:06.288115 32404 fs_manager.cc:359] Metadata directory not provided
I20260811 12:06:06.288245 32404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:06:06.294137 32404 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.005s
I20260811 12:06:06.297104 32426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:06:06.298463 32404 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 12:06:06.298614 32404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "b26f2b577ada48839a0741a87dd73acc"
format_stamp: "Formatted at 2026-08-11 12:06:03 on dist-test-slave-mjjr"
server_key: "5a61fb635d34e6b1e61380c5b17ea827"
server_key_iv: "a76de3da28dd8119ae7c728dd9eec038"
server_key_version: "encryptionkey@0"
I20260811 12:06:06.299713 32404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:06:06.323246 32404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:06:06.328042 32404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:06:06.328284 32404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:06:06.347476 32404 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.190:32893
I20260811 12:06:06.347500 32508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.190:32893 every 8 connection(s)
I20260811 12:06:06.349128 32404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:06:06.350248 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 32404
I20260811 12:06:06.351140 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.129:43519
--local_ip_for_outbound_sockets=127.9.221.129
--tserver_master_addrs=127.9.221.190:32893
--webserver_port=43271
--webserver_interface=127.9.221.129
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab
--principal=kudu/127.9.221.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:37169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:06:06.356626 32509 sys_catalog.cc:263] Verifying existing consensus state
I20260811 12:06:06.359983 32509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc: Bootstrap starting.
I20260811 12:06:06.388072 32509 log.cc:824] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc: Log is configured to *not* fsync() on all Append() calls
I20260811 12:06:06.403750 32509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 12:06:06.404414 32509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc: Bootstrap complete.
I20260811 12:06:06.418467 32509 raft_consensus.cc:348] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b26f2b577ada48839a0741a87dd73acc" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 32893 } }
I20260811 12:06:06.419684 32509 raft_consensus.cc:729] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b26f2b577ada48839a0741a87dd73acc, State: Initialized, Role: FOLLOWER
I20260811 12:06:06.420357 32509 consensus_queue.cc:260] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b26f2b577ada48839a0741a87dd73acc" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 32893 } }
I20260811 12:06:06.420655 32509 raft_consensus.cc:388] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:06:06.420778 32509 raft_consensus.cc:482] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:06:06.420962 32509 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [term 1 FOLLOWER]: Advancing to term 2
I20260811 12:06:06.427546 32509 raft_consensus.cc:504] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b26f2b577ada48839a0741a87dd73acc" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 32893 } }
I20260811 12:06:06.428212 32509 leader_election.cc:302] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b26f2b577ada48839a0741a87dd73acc; no voters:
I20260811 12:06:06.428861 32509 leader_election.cc:288] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 12:06:06.429018 32515 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [term 2 FOLLOWER]: Leader election won for term 2
I20260811 12:06:06.430413 32515 raft_consensus.cc:686] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [term 2 LEADER]: Becoming Leader. State: Replica: b26f2b577ada48839a0741a87dd73acc, State: Running, Role: LEADER
I20260811 12:06:06.430936 32515 consensus_queue.cc:237] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b26f2b577ada48839a0741a87dd73acc" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 32893 } }
I20260811 12:06:06.431329 32509 sys_catalog.cc:564] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:06:06.433368 32519 sys_catalog.cc:455] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b26f2b577ada48839a0741a87dd73acc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b26f2b577ada48839a0741a87dd73acc" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 32893 } } }
I20260811 12:06:06.434134 32519 sys_catalog.cc:458] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [sys.catalog]: This master's current role is: LEADER
I20260811 12:06:06.433701 32520 sys_catalog.cc:455] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [sys.catalog]: SysCatalogTable state changed. Reason: New leader b26f2b577ada48839a0741a87dd73acc. Latest consensus state: current_term: 2 leader_uuid: "b26f2b577ada48839a0741a87dd73acc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b26f2b577ada48839a0741a87dd73acc" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 32893 } } }
I20260811 12:06:06.434787 32520 sys_catalog.cc:458] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [sys.catalog]: This master's current role is: LEADER
I20260811 12:06:06.437959 32525 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:06:06.445881 32525 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:06:06.447295 32525 catalog_manager.cc:1259] Loaded cluster ID: 0441414dd2804a6984a06b101f730a8c
I20260811 12:06:06.447405 32525 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:06:06.450827 32525 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260811 12:06:06.468957 32525 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260811 12:06:06.489566 32525 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:06:06.491324 32525 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc: Loaded TSK: 0
I20260811 12:06:06.493311 32525 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 12:06:06.733568 32512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:06:06.733837 32512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:06:06.733979 32512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:06:06.741050 32512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:06:06.741148 32512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:06:06.741302 32512 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.129
I20260811 12:06:06.750855 32512 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:37169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab
--principal=kudu/127.9.221.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.129:43519
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.221.129
--webserver_port=43271
--tserver_master_addrs=127.9.221.190:32893
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.129
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:06:06.752188 32512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:06:06.753501 32512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:06:06.768294 32556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:06:06.768551 32559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:06:06.769127 32562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:06:06.769977 32512 server_base.cc:1034] running on GCE node
I20260811 12:06:06.770720 32512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:06:06.771950 32512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:06:06.773207 32512 hybrid_clock.cc:648] HybridClock initialized: now 1786449966773156 us; error 56 us; skew 500 ppm
I20260811 12:06:06.773583 32512 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:06:06 dist-test-slave-mjjr krb5kdc[31590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449966, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:06:06.777369 32512 init.cc:377] Logged in from keytab as kudu/127.9.221.129@KRBTEST.COM (short username kudu)
I20260811 12:06:06.778847 32512 webserver.cc:460] Webserver started at http://127.9.221.129:43271/ using document root <none> and password file <none>
I20260811 12:06:06.779524 32512 fs_manager.cc:359] Metadata directory not provided
I20260811 12:06:06.779719 32512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:06:06.785167 32512 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.001s
I20260811 12:06:06.787947 32571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:06:06.789569 32512 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 12:06:06.789731 32512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-0/wal
uuid: "380d1e238e15456d98dee6b99cdf4f78"
format_stamp: "Formatted at 2026-08-11 12:06:03 on dist-test-slave-mjjr"
server_key: "aedf699819165298ab6f5baee3205207"
server_key_iv: "1ea069fc7f6394608d75adb016ee9668"
server_key_version: "encryptionkey@0"
I20260811 12:06:06.790697 32512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:06:06.811722 32512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:06:06.816108 32512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:06:06.816404 32512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:06:06.817438 32512 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:06:06.818953 32512 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:06:06.819069 32512 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:06:06.819188 32512 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:06:06.819283 32512 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:06:06.854223 32512 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.129:43519
I20260811 12:06:06.854254 32753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.129:43519 every 8 connection(s)
I20260811 12:06:06.855674 32512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
I20260811 12:06:06.864900 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 32512
I20260811 12:06:06.865672 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.130:44905
--local_ip_for_outbound_sockets=127.9.221.130
--tserver_master_addrs=127.9.221.190:32893
--webserver_port=41937
--webserver_interface=127.9.221.130
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab
--principal=kudu/127.9.221.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:37169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 12:06:06 dist-test-slave-mjjr krb5kdc[31590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449966, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:06:06.880748 32759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:06.858527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:37757 (local address 127.9.221.190:32893)
0811 12:06:06.859135 (+ 608us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:06.859146 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:06.859875 (+ 729us) server_negotiation.cc:407] Connection header received
0811 12:06:06.861880 (+ 2005us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:06.861891 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:06.862201 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:06.862478 (+ 277us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:06:06.863876 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:06.865884 (+ 2008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:06.867533 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:06.868635 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:06.868839 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:06.872164 (+ 3325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:06:06.872178 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:06:06.872196 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:06:06.872244 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:06:06.876261 (+ 4017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:06.877259 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:06.877265 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:06.877270 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:06.877421 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:06.877933 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:06.877936 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:06.877938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:06.878370 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:06:06.878521 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:06.879400 (+ 879us) server_negotiation.cc:299] Negotiation successful
0811 12:06:06.879776 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":270,"threads_started":1}
I20260811 12:06:06.881232 32760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:06.858776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.129:37757)
0811 12:06:06.859712 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:06.859757 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 12:06:06.861424 (+ 1667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:06:06.862612 (+ 1188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:06:06.862619 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:06:06.863079 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:06:06.863613 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:06.863630 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:06.866102 (+ 2472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:06.866110 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:06.867377 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:06.867389 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:06.868842 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:06.868847 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:06.869026 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:06.869704 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:06:06.869730 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:06:06.871888 (+ 2158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:06:06.876409 (+ 4521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:06:06.876421 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:06:06.876437 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:06:06.877114 (+ 677us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:06:06.877542 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:06:06.877550 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:06:06.877556 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:06:06.877816 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:06:06.878506 (+ 690us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:06:06.878516 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:06:06.878940 (+ 424us) client_negotiation.cc:769] Sending connection context
0811 12:06:06.879223 (+ 283us) client_negotiation.cc:240] Negotiation successful
0811 12:06:06.879817 (+ 594us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":374,"threads_started":1}
I20260811 12:06:06.884207 32756 heartbeater.cc:347] Connected to a master server at 127.9.221.190:32893
I20260811 12:06:06.884503 32756 heartbeater.cc:464] Registering TS with master...
I20260811 12:06:06.885227 32756 heartbeater.cc:511] Master 127.9.221.190:32893 requested a full tablet report, sending...
I20260811 12:06:06.887557 32448 ts_manager.cc:194] Registered new tserver with Master: 380d1e238e15456d98dee6b99cdf4f78 (127.9.221.129:43519)
I20260811 12:06:06.889967 32448 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.129@KRBTEST.COM'} at 127.9.221.129:37757
W20260811 12:06:07.220530 32762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:06:07.220860 32762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:06:07.220964 32762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:06:07.228396 32762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:06:07.228518 32762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:06:07.228652 32762 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.130
I20260811 12:06:07.238606 32762 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:37169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab
--principal=kudu/127.9.221.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.130:44905
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.221.130
--webserver_port=41937
--tserver_master_addrs=127.9.221.190:32893
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.130
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:06:07.239868 32762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:06:07.241197 32762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:06:07.255152 303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:06:07.255198 306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:06:07.255241 308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:06:07.256194 32762 server_base.cc:1034] running on GCE node
I20260811 12:06:07.256795 32762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:06:07.257756 32762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:06:07.258977 32762 hybrid_clock.cc:648] HybridClock initialized: now 1786449967258924 us; error 44 us; skew 500 ppm
I20260811 12:06:07.259346 32762 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:06:07 dist-test-slave-mjjr krb5kdc[31590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449967, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:06:07.262842 32762 init.cc:377] Logged in from keytab as kudu/127.9.221.130@KRBTEST.COM (short username kudu)
I20260811 12:06:07.264557 32762 webserver.cc:460] Webserver started at http://127.9.221.130:41937/ using document root <none> and password file <none>
I20260811 12:06:07.265354 32762 fs_manager.cc:359] Metadata directory not provided
I20260811 12:06:07.265517 32762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:06:07.270818 32762 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260811 12:06:07.273563 321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:06:07.274945 32762 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 12:06:07.275120 32762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-1/wal
uuid: "43d08b9a29cf42a48adb34153409186a"
format_stamp: "Formatted at 2026-08-11 12:06:04 on dist-test-slave-mjjr"
server_key: "f0f3ff1b9439b238d8bea92bddd09620"
server_key_iv: "ecca2f464b5cf5fc19d63d5ef807c261"
server_key_version: "encryptionkey@0"
I20260811 12:06:07.276189 32762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:06:07.290812 32762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:06:07.295184 32762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:06:07.295406 32762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:06:07.296365 32762 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:06:07.297737 32762 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:06:07.297824 32762 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:06:07.297899 32762 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:06:07.297964 32762 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:06:07.330369 32762 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.130:44905
I20260811 12:06:07.330458 493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.130:44905 every 8 connection(s)
I20260811 12:06:07.332062 32762 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
I20260811 12:06:07.338124 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 32762
I20260811 12:06:07.338866 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.131:38705
--local_ip_for_outbound_sockets=127.9.221.131
--tserver_master_addrs=127.9.221.190:32893
--webserver_port=37811
--webserver_interface=127.9.221.131
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab
--principal=kudu/127.9.221.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:37169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 12:06:07 dist-test-slave-mjjr krb5kdc[31590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449967, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.130@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:06:07.353554 32759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:07.334684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:54147 (local address 127.9.221.190:32893)
0811 12:06:07.334862 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:07.334867 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:07.336100 (+ 1233us) server_negotiation.cc:407] Connection header received
0811 12:06:07.337827 (+ 1727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:07.337836 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:07.337939 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:07.338073 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:06:07.339668 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:07.340851 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:07.342361 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:07.343090 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:07.343237 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:07.346603 (+ 3366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:06:07.346613 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:06:07.346620 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:06:07.346667 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:06:07.350047 (+ 3380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:07.350774 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:07.350781 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:07.350784 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:07.350928 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:07.351387 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:07.351395 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:07.351400 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:07.351830 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:06:07.351997 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:07.352818 (+ 821us) server_negotiation.cc:299] Negotiation successful
0811 12:06:07.353090 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 12:06:07.354563 501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:07.335186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.130:54147)
0811 12:06:07.335942 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:07.335985 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 12:06:07.337564 (+ 1579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:06:07.338328 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:06:07.338336 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:06:07.338842 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:06:07.339467 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:07.339485 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:07.341006 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:07.341009 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:07.342194 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:07.342206 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:07.343292 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:07.343296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:07.343487 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:07.344227 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:06:07.344249 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:06:07.346352 (+ 2103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:06:07.350191 (+ 3839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:06:07.350201 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:06:07.350214 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:06:07.350643 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:06:07.351058 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:06:07.351063 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:06:07.351067 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:06:07.351254 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:06:07.351971 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:06:07.351979 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:06:07.352353 (+ 374us) client_negotiation.cc:769] Sending connection context
0811 12:06:07.352676 (+ 323us) client_negotiation.cc:240] Negotiation successful
0811 12:06:07.353161 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":271,"threads_started":1}
I20260811 12:06:07.356031 495 heartbeater.cc:347] Connected to a master server at 127.9.221.190:32893
I20260811 12:06:07.356346 495 heartbeater.cc:464] Registering TS with master...
I20260811 12:06:07.357198 495 heartbeater.cc:511] Master 127.9.221.190:32893 requested a full tablet report, sending...
I20260811 12:06:07.358841 32448 ts_manager.cc:194] Registered new tserver with Master: 43d08b9a29cf42a48adb34153409186a (127.9.221.130:44905)
I20260811 12:06:07.359965 32448 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.130@KRBTEST.COM'} at 127.9.221.130:54147
W20260811 12:06:07.678246 502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:06:07.678577 502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:06:07.678748 502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:06:07.685935 502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:06:07.686097 502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:06:07.686283 502 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.131
I20260811 12:06:07.696195 502 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:37169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab
--principal=kudu/127.9.221.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.131:38705
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.221.131
--webserver_port=37811
--tserver_master_addrs=127.9.221.190:32893
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.131
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:06:07.697660 502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:06:07.698989 502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:06:07.712889 507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:06:07.712913 514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:06:07.712882 511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:06:07.713613 502 server_base.cc:1034] running on GCE node
I20260811 12:06:07.714285 502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:06:07.715241 502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:06:07.716473 502 hybrid_clock.cc:648] HybridClock initialized: now 1786449967716416 us; error 60 us; skew 500 ppm
I20260811 12:06:07.716787 502 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:06:07 dist-test-slave-mjjr krb5kdc[31590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449967, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:06:07.719878 502 init.cc:377] Logged in from keytab as kudu/127.9.221.131@KRBTEST.COM (short username kudu)
I20260811 12:06:07.721367 502 webserver.cc:460] Webserver started at http://127.9.221.131:37811/ using document root <none> and password file <none>
I20260811 12:06:07.722079 502 fs_manager.cc:359] Metadata directory not provided
I20260811 12:06:07.722242 502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:06:07.727536 502 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:06:07.730103 523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:06:07.731387 502 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:06:07.731576 502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-2/wal
uuid: "d7bf3363e68843e189b30a08dde3b52d"
format_stamp: "Formatted at 2026-08-11 12:06:04 on dist-test-slave-mjjr"
server_key: "5a0fef63527135fba81646238b899a8a"
server_key_iv: "4116aa837c816cac98532681b80e9e35"
server_key_version: "encryptionkey@0"
I20260811 12:06:07.732695 502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:06:07.763523 502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:06:07.767771 502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:06:07.768059 502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:06:07.769053 502 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:06:07.770440 502 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:06:07.770577 502 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:06:07.770706 502 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:06:07.770797 502 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:06:07.805754 502 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.131:38705
I20260811 12:06:07.805811 700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.131:38705 every 8 connection(s)
I20260811 12:06:07.807248 502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
I20260811 12:06:07.810918 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 502
Aug 11 12:06:07 dist-test-slave-mjjr krb5kdc[31590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449967, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.131@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:06:07.829674 32759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:07.809754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:40611 (local address 127.9.221.190:32893)
0811 12:06:07.809961 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:07.809966 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:07.811312 (+ 1346us) server_negotiation.cc:407] Connection header received
0811 12:06:07.813414 (+ 2102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:07.813423 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:07.813545 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:07.813690 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:06:07.815143 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:07.816747 (+ 1604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:07.818293 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:07.818896 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:07.819047 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:07.822664 (+ 3617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:06:07.822672 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:06:07.822676 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:06:07.822715 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:06:07.826203 (+ 3488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:07.827060 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:07.827067 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:07.827074 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:07.827235 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:07.827780 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:07.827784 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:07.827786 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:07.828066 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:06:07.828242 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:07.829151 (+ 909us) server_negotiation.cc:299] Negotiation successful
0811 12:06:07.829392 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 12:06:07.831067 704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:07.810280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.131:40611)
0811 12:06:07.811133 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:07.811177 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 12:06:07.813116 (+ 1939us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:06:07.813864 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:06:07.813871 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:06:07.814346 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:06:07.814976 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:07.814993 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:07.816918 (+ 1925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:07.816922 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:07.818153 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:07.818164 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:07.819141 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:07.819149 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:07.819387 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:07.820119 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:06:07.820141 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:06:07.822449 (+ 2308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:06:07.826356 (+ 3907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:06:07.826362 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:06:07.826373 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:06:07.826939 (+ 566us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:06:07.827369 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:06:07.827375 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:06:07.827377 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:06:07.827668 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:06:07.828220 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:06:07.828226 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:06:07.828715 (+ 489us) client_negotiation.cc:769] Sending connection context
0811 12:06:07.829076 (+ 361us) client_negotiation.cc:240] Negotiation successful
0811 12:06:07.829630 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":379,"threads_started":1}
I20260811 12:06:07.832599 701 heartbeater.cc:347] Connected to a master server at 127.9.221.190:32893
I20260811 12:06:07.832907 701 heartbeater.cc:464] Registering TS with master...
I20260811 12:06:07.833846 701 heartbeater.cc:511] Master 127.9.221.190:32893 requested a full tablet report, sending...
I20260811 12:06:07.834304 705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:07.811929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55246 (local address 127.9.221.190:32893)
0811 12:06:07.812490 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:07.812499 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:07.818773 (+ 6274us) server_negotiation.cc:407] Connection header received
0811 12:06:07.818964 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:07.818969 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:07.819035 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:07.819148 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:06:07.820905 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:07.821965 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:07.823627 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:07.824171 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:07.824274 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:07.827495 (+ 3221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:06:07.827503 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:06:07.827509 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:06:07.827542 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:06:07.830945 (+ 3403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:07.832016 (+ 1071us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:07.832019 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:07.832021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:07.832184 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:07.832772 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:07.832776 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:07.832778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:07.833064 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:06:07.833299 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:07.833634 (+ 335us) server_negotiation.cc:299] Negotiation successful
0811 12:06:07.833880 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":259,"threads_started":1}
I20260811 12:06:07.835448 32448 ts_manager.cc:194] Registered new tserver with Master: d7bf3363e68843e189b30a08dde3b52d (127.9.221.131:38705)
I20260811 12:06:07.836499 32448 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.131@KRBTEST.COM'} at 127.9.221.131:40611
I20260811 12:06:07.837116 10102 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:06:07.843437 705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:07.838083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55252 (local address 127.9.221.190:32893)
0811 12:06:07.838264 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:07.838272 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:07.838742 (+ 470us) server_negotiation.cc:407] Connection header received
0811 12:06:07.838935 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:07.838939 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:07.839059 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:07.839204 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:06:07.839585 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:07.840305 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:07.841608 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:07.842008 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:07.842097 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:07.842510 (+ 413us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:06:07.842919 (+ 409us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:06:07.843039 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:07.843112 (+ 73us) server_negotiation.cc:299] Negotiation successful
0811 12:06:07.843166 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 12:06:07.844310 32448 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:55252:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:06:07.851783 32448 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:06:07.885277 714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:07.864041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45250 (local address 127.9.221.131:38705)
0811 12:06:07.875729 (+ 11688us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:07.875737 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:07.875778 (+ 41us) server_negotiation.cc:407] Connection header received
0811 12:06:07.875891 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:07.875898 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:07.876161 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:07.877411 (+ 1250us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:06:07.877740 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:07.879898 (+ 2158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:07.882511 (+ 2613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:07.883972 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:07.884109 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:07.884353 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:07.884742 (+ 389us) server_negotiation.cc:299] Negotiation successful
0811 12:06:07.884915 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11478,"thread_start_us":11280,"threads_started":1}
I20260811 12:06:07.885507 716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:07.863736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.131:38705 (local address 127.0.0.1:45250)
0811 12:06:07.870532 (+ 6796us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:07.870578 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 12:06:07.870902 (+ 324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:06:07.876474 (+ 5572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:06:07.876485 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:06:07.876541 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:06:07.877121 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:07.877138 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:07.880071 (+ 2933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:07.880077 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:07.882310 (+ 2233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:07.882324 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:07.884206 (+ 1882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:07.884213 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:07.884481 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:07.884533 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 12:06:07.884676 (+ 143us) client_negotiation.cc:240] Negotiation successful
0811 12:06:07.884884 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6476,"mutex_wait_us":51,"thread_start_us":17812,"threads_started":1}
I20260811 12:06:07.888847 721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:07.864872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:43519 (local address 127.0.0.1:42162)
0811 12:06:07.876077 (+ 11205us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:07.876091 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 12:06:07.876179 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:06:07.881075 (+ 4896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:06:07.881087 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:06:07.881115 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:06:07.881867 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:07.881886 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:07.883863 (+ 1977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:07.883869 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:07.886042 (+ 2173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:07.886059 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:07.888020 (+ 1961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:07.888026 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:07.888183 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:07.888214 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 12:06:07.888317 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 12:06:07.888445 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11110,"mutex_wait_us":31,"thread_start_us":6244,"threads_started":1}
I20260811 12:06:07.889566 715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:07.864236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42162 (local address 127.9.221.129:43519)
0811 12:06:07.880315 (+ 16079us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:07.880326 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:07.880361 (+ 35us) server_negotiation.cc:407] Connection header received
0811 12:06:07.880485 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:07.880497 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:07.880742 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:07.881084 (+ 342us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:06:07.882074 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:07.883667 (+ 1593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:07.886206 (+ 2539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:07.887439 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:07.887578 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:07.887844 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:07.889084 (+ 1240us) server_negotiation.cc:299] Negotiation successful
0811 12:06:07.889248 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15877,"thread_start_us":15705,"threads_started":1}
I20260811 12:06:07.889673 713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:07.873823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.130:44905 (local address 127.0.0.1:46480)
0811 12:06:07.881890 (+ 8067us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:07.881907 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:06:07.882102 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:06:07.882950 (+ 848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:06:07.882954 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:06:07.882972 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:06:07.883252 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:07.883258 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:07.884873 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:07.884877 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:07.887049 (+ 2172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:07.887061 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:07.888812 (+ 1751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:07.888819 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:07.889019 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:07.889047 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 12:06:07.889146 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 12:06:07.889305 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7929,"mutex_wait_us":5,"thread_start_us":2037,"threads_started":1}
I20260811 12:06:07.889799 719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:07.873267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46480 (local address 127.9.221.130:44905)
0811 12:06:07.873837 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:07.873848 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:07.882006 (+ 8158us) server_negotiation.cc:407] Connection header received
0811 12:06:07.882306 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:07.882315 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:07.882560 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:07.882743 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:06:07.883417 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:07.884717 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:07.887203 (+ 2486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:07.888567 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:07.888719 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:07.889114 (+ 395us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:07.889193 (+ 79us) server_negotiation.cc:299] Negotiation successful
0811 12:06:07.889353 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":285,"threads_started":1}
I20260811 12:06:07.889876 590 tablet_service.cc:1467] Processing CreateTablet for tablet c0d702d4922d4af4817459f7889e5d08 (DEFAULT_TABLE table=test-table [id=3e48426511814b3ca825422d0f4a2caf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:06:07.891470 590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0d702d4922d4af4817459f7889e5d08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:06:07.893190 32756 heartbeater.cc:503] Master 127.9.221.190:32893 was elected leader, sending a full tablet report...
I20260811 12:06:07.892758 391 tablet_service.cc:1467] Processing CreateTablet for tablet c0d702d4922d4af4817459f7889e5d08 (DEFAULT_TABLE table=test-table [id=3e48426511814b3ca825422d0f4a2caf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:06:07.893471 32648 tablet_service.cc:1467] Processing CreateTablet for tablet c0d702d4922d4af4817459f7889e5d08 (DEFAULT_TABLE table=test-table [id=3e48426511814b3ca825422d0f4a2caf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:06:07.894066 391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0d702d4922d4af4817459f7889e5d08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:06:07.894773 32648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0d702d4922d4af4817459f7889e5d08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:06:07.908524 727 tablet_bootstrap.cc:492] T c0d702d4922d4af4817459f7889e5d08 P d7bf3363e68843e189b30a08dde3b52d: Bootstrap starting.
I20260811 12:06:07.910267 728 tablet_bootstrap.cc:492] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78: Bootstrap starting.
I20260811 12:06:07.911669 726 tablet_bootstrap.cc:492] T c0d702d4922d4af4817459f7889e5d08 P 43d08b9a29cf42a48adb34153409186a: Bootstrap starting.
I20260811 12:06:07.912979 727 tablet_bootstrap.cc:654] T c0d702d4922d4af4817459f7889e5d08 P d7bf3363e68843e189b30a08dde3b52d: Neither blocks nor log segments found. Creating new log.
I20260811 12:06:07.913828 728 tablet_bootstrap.cc:654] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78: Neither blocks nor log segments found. Creating new log.
I20260811 12:06:07.914143 727 log.cc:824] T c0d702d4922d4af4817459f7889e5d08 P d7bf3363e68843e189b30a08dde3b52d: Log is configured to *not* fsync() on all Append() calls
I20260811 12:06:07.915118 726 tablet_bootstrap.cc:654] T c0d702d4922d4af4817459f7889e5d08 P 43d08b9a29cf42a48adb34153409186a: Neither blocks nor log segments found. Creating new log.
I20260811 12:06:07.915172 728 log.cc:824] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78: Log is configured to *not* fsync() on all Append() calls
I20260811 12:06:07.916436 726 log.cc:824] T c0d702d4922d4af4817459f7889e5d08 P 43d08b9a29cf42a48adb34153409186a: Log is configured to *not* fsync() on all Append() calls
I20260811 12:06:07.917822 727 tablet_bootstrap.cc:492] T c0d702d4922d4af4817459f7889e5d08 P d7bf3363e68843e189b30a08dde3b52d: No bootstrap required, opened a new log
I20260811 12:06:07.918130 727 ts_tablet_manager.cc:1397] T c0d702d4922d4af4817459f7889e5d08 P d7bf3363e68843e189b30a08dde3b52d: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260811 12:06:07.918723 728 tablet_bootstrap.cc:492] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78: No bootstrap required, opened a new log
I20260811 12:06:07.918987 728 ts_tablet_manager.cc:1397] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260811 12:06:07.919222 726 tablet_bootstrap.cc:492] T c0d702d4922d4af4817459f7889e5d08 P 43d08b9a29cf42a48adb34153409186a: No bootstrap required, opened a new log
I20260811 12:06:07.919471 726 ts_tablet_manager.cc:1397] T c0d702d4922d4af4817459f7889e5d08 P 43d08b9a29cf42a48adb34153409186a: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260811 12:06:07.926164 726 raft_consensus.cc:348] T c0d702d4922d4af4817459f7889e5d08 P 43d08b9a29cf42a48adb34153409186a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7bf3363e68843e189b30a08dde3b52d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 38705 } } peers { permanent_uuid: "380d1e238e15456d98dee6b99cdf4f78" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43519 } } peers { permanent_uuid: "43d08b9a29cf42a48adb34153409186a" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 44905 } }
I20260811 12:06:07.926555 726 raft_consensus.cc:374] T c0d702d4922d4af4817459f7889e5d08 P 43d08b9a29cf42a48adb34153409186a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:06:07.926688 726 raft_consensus.cc:729] T c0d702d4922d4af4817459f7889e5d08 P 43d08b9a29cf42a48adb34153409186a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43d08b9a29cf42a48adb34153409186a, State: Initialized, Role: FOLLOWER
I20260811 12:06:07.927377 726 consensus_queue.cc:260] T c0d702d4922d4af4817459f7889e5d08 P 43d08b9a29cf42a48adb34153409186a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7bf3363e68843e189b30a08dde3b52d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 38705 } } peers { permanent_uuid: "380d1e238e15456d98dee6b99cdf4f78" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43519 } } peers { permanent_uuid: "43d08b9a29cf42a48adb34153409186a" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 44905 } }
I20260811 12:06:07.928890 495 heartbeater.cc:503] Master 127.9.221.190:32893 was elected leader, sending a full tablet report...
I20260811 12:06:07.929019 727 raft_consensus.cc:348] T c0d702d4922d4af4817459f7889e5d08 P d7bf3363e68843e189b30a08dde3b52d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7bf3363e68843e189b30a08dde3b52d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 38705 } } peers { permanent_uuid: "380d1e238e15456d98dee6b99cdf4f78" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43519 } } peers { permanent_uuid: "43d08b9a29cf42a48adb34153409186a" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 44905 } }
I20260811 12:06:07.929419 727 raft_consensus.cc:374] T c0d702d4922d4af4817459f7889e5d08 P d7bf3363e68843e189b30a08dde3b52d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:06:07.929519 727 raft_consensus.cc:729] T c0d702d4922d4af4817459f7889e5d08 P d7bf3363e68843e189b30a08dde3b52d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7bf3363e68843e189b30a08dde3b52d, State: Initialized, Role: FOLLOWER
I20260811 12:06:07.929538 726 ts_tablet_manager.cc:1428] T c0d702d4922d4af4817459f7889e5d08 P 43d08b9a29cf42a48adb34153409186a: Time spent starting tablet: real 0.010s user 0.009s sys 0.001s
I20260811 12:06:07.929384 728 raft_consensus.cc:348] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7bf3363e68843e189b30a08dde3b52d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 38705 } } peers { permanent_uuid: "380d1e238e15456d98dee6b99cdf4f78" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43519 } } peers { permanent_uuid: "43d08b9a29cf42a48adb34153409186a" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 44905 } }
I20260811 12:06:07.929787 728 raft_consensus.cc:374] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:06:07.929885 728 raft_consensus.cc:729] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 380d1e238e15456d98dee6b99cdf4f78, State: Initialized, Role: FOLLOWER
I20260811 12:06:07.930173 727 consensus_queue.cc:260] T c0d702d4922d4af4817459f7889e5d08 P d7bf3363e68843e189b30a08dde3b52d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7bf3363e68843e189b30a08dde3b52d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 38705 } } peers { permanent_uuid: "380d1e238e15456d98dee6b99cdf4f78" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43519 } } peers { permanent_uuid: "43d08b9a29cf42a48adb34153409186a" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 44905 } }
I20260811 12:06:07.930495 728 consensus_queue.cc:260] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7bf3363e68843e189b30a08dde3b52d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 38705 } } peers { permanent_uuid: "380d1e238e15456d98dee6b99cdf4f78" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43519 } } peers { permanent_uuid: "43d08b9a29cf42a48adb34153409186a" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 44905 } }
I20260811 12:06:07.931308 701 heartbeater.cc:503] Master 127.9.221.190:32893 was elected leader, sending a full tablet report...
I20260811 12:06:07.931985 727 ts_tablet_manager.cc:1428] T c0d702d4922d4af4817459f7889e5d08 P d7bf3363e68843e189b30a08dde3b52d: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260811 12:06:07.933715 728 ts_tablet_manager.cc:1428] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78: Time spent starting tablet: real 0.015s user 0.013s sys 0.001s
I20260811 12:06:07.986061 734 raft_consensus.cc:482] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:06:07.986338 734 raft_consensus.cc:504] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7bf3363e68843e189b30a08dde3b52d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 38705 } } peers { permanent_uuid: "380d1e238e15456d98dee6b99cdf4f78" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43519 } } peers { permanent_uuid: "43d08b9a29cf42a48adb34153409186a" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 44905 } }
I20260811 12:06:07.988449 734 leader_election.cc:288] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7bf3363e68843e189b30a08dde3b52d (127.9.221.131:38705), 43d08b9a29cf42a48adb34153409186a (127.9.221.130:44905)
I20260811 12:06:07.995209 714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:07.988863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:46953 (local address 127.9.221.131:38705)
0811 12:06:07.989116 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:07.989124 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:07.989819 (+ 695us) server_negotiation.cc:407] Connection header received
0811 12:06:07.990011 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:07.990021 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:07.990080 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:07.990191 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:06:07.990812 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:07.991521 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:07.993309 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:07.994176 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:07.994356 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:07.994470 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:07.994785 (+ 315us) server_negotiation.cc:299] Negotiation successful
0811 12:06:07.994926 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 12:06:07.995221 719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:07.989094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:34079 (local address 127.9.221.130:44905)
0811 12:06:07.989285 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:07.989290 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:07.989697 (+ 407us) server_negotiation.cc:407] Connection header received
0811 12:06:07.989935 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:07.989939 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:07.989999 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:07.990094 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:06:07.990733 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:07.991422 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:07.993280 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:07.994170 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:07.994335 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:07.994421 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:07.994801 (+ 380us) server_negotiation.cc:299] Negotiation successful
0811 12:06:07.994936 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 12:06:07.995389 736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:07.988853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.131:38705 (local address 127.9.221.129:46953)
0811 12:06:07.989708 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:07.989727 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:06:07.989873 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:06:07.990205 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:06:07.990214 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:06:07.990238 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:06:07.990643 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:07.990652 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:07.991686 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:07.991698 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:07.993155 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:07.993167 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:07.994463 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:07.994469 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:07.994606 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:07.994647 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 12:06:07.994748 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 12:06:07.994947 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":757,"mutex_wait_us":74,"thread_start_us":546,"threads_started":1}
I20260811 12:06:07.995442 735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:07.988714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.130:44905 (local address 127.9.221.129:34079)
0811 12:06:07.989594 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:07.989623 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 12:06:07.989775 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:06:07.990115 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:06:07.990119 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:06:07.990150 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:06:07.990580 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:07.990590 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:07.991554 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:07.991561 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:07.993103 (+ 1542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:07.993133 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:07.994411 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:07.994418 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:07.994596 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:07.994671 (+ 75us) client_negotiation.cc:769] Sending connection context
0811 12:06:07.994773 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 12:06:07.995019 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"mutex_wait_us":39,"spinlock_wait_cycles":22272,"thread_start_us":417,"threads_started":1}
I20260811 12:06:07.996491 425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0d702d4922d4af4817459f7889e5d08" candidate_uuid: "380d1e238e15456d98dee6b99cdf4f78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43d08b9a29cf42a48adb34153409186a" is_pre_election: true
I20260811 12:06:07.996486 622 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0d702d4922d4af4817459f7889e5d08" candidate_uuid: "380d1e238e15456d98dee6b99cdf4f78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7bf3363e68843e189b30a08dde3b52d" is_pre_election: true
I20260811 12:06:07.997026 622 raft_consensus.cc:2454] T c0d702d4922d4af4817459f7889e5d08 P d7bf3363e68843e189b30a08dde3b52d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 380d1e238e15456d98dee6b99cdf4f78 in term 0.
I20260811 12:06:07.997025 425 raft_consensus.cc:2454] T c0d702d4922d4af4817459f7889e5d08 P 43d08b9a29cf42a48adb34153409186a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 380d1e238e15456d98dee6b99cdf4f78 in term 0.
I20260811 12:06:07.997685 32580 leader_election.cc:302] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 380d1e238e15456d98dee6b99cdf4f78, 43d08b9a29cf42a48adb34153409186a; no voters:
I20260811 12:06:07.998081 734 raft_consensus.cc:2781] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:06:07.998230 734 raft_consensus.cc:482] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:06:07.998354 734 raft_consensus.cc:3037] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:06:08.000614 734 raft_consensus.cc:504] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7bf3363e68843e189b30a08dde3b52d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 38705 } } peers { permanent_uuid: "380d1e238e15456d98dee6b99cdf4f78" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43519 } } peers { permanent_uuid: "43d08b9a29cf42a48adb34153409186a" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 44905 } }
I20260811 12:06:08.001327 734 leader_election.cc:288] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [CANDIDATE]: Term 1 election: Requested vote from peers d7bf3363e68843e189b30a08dde3b52d (127.9.221.131:38705), 43d08b9a29cf42a48adb34153409186a (127.9.221.130:44905)
I20260811 12:06:08.001724 622 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0d702d4922d4af4817459f7889e5d08" candidate_uuid: "380d1e238e15456d98dee6b99cdf4f78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7bf3363e68843e189b30a08dde3b52d"
I20260811 12:06:08.001832 425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0d702d4922d4af4817459f7889e5d08" candidate_uuid: "380d1e238e15456d98dee6b99cdf4f78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43d08b9a29cf42a48adb34153409186a"
I20260811 12:06:08.001932 622 raft_consensus.cc:3037] T c0d702d4922d4af4817459f7889e5d08 P d7bf3363e68843e189b30a08dde3b52d [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:06:08.002023 425 raft_consensus.cc:3037] T c0d702d4922d4af4817459f7889e5d08 P 43d08b9a29cf42a48adb34153409186a [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:06:08.005330 622 raft_consensus.cc:2454] T c0d702d4922d4af4817459f7889e5d08 P d7bf3363e68843e189b30a08dde3b52d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 380d1e238e15456d98dee6b99cdf4f78 in term 1.
I20260811 12:06:08.005326 425 raft_consensus.cc:2454] T c0d702d4922d4af4817459f7889e5d08 P 43d08b9a29cf42a48adb34153409186a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 380d1e238e15456d98dee6b99cdf4f78 in term 1.
I20260811 12:06:08.005757 32577 leader_election.cc:302] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 380d1e238e15456d98dee6b99cdf4f78, d7bf3363e68843e189b30a08dde3b52d; no voters:
I20260811 12:06:08.006060 734 raft_consensus.cc:2781] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:06:08.006531 734 raft_consensus.cc:686] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [term 1 LEADER]: Becoming Leader. State: Replica: 380d1e238e15456d98dee6b99cdf4f78, State: Running, Role: LEADER
I20260811 12:06:08.007048 734 consensus_queue.cc:237] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7bf3363e68843e189b30a08dde3b52d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 38705 } } peers { permanent_uuid: "380d1e238e15456d98dee6b99cdf4f78" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43519 } } peers { permanent_uuid: "43d08b9a29cf42a48adb34153409186a" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 44905 } }
I20260811 12:06:08.011788 32447 catalog_manager.cc:5495] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 380d1e238e15456d98dee6b99cdf4f78 (127.9.221.129). New cstate: current_term: 1 leader_uuid: "380d1e238e15456d98dee6b99cdf4f78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7bf3363e68843e189b30a08dde3b52d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 38705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "380d1e238e15456d98dee6b99cdf4f78" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "43d08b9a29cf42a48adb34153409186a" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 44905 } health_report { overall_health: UNKNOWN } } }
I20260811 12:06:08.029203 715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:08.023607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42178 (local address 127.9.221.129:43519)
0811 12:06:08.023818 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:08.023824 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:08.023925 (+ 101us) server_negotiation.cc:407] Connection header received
0811 12:06:08.024174 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:08.024178 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:08.024248 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:08.024399 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:06:08.024806 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:08.025516 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:08.026954 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:08.027480 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:08.027613 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:08.028157 (+ 544us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:06:08.028619 (+ 462us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:06:08.028786 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:08.028884 (+ 98us) server_negotiation.cc:299] Negotiation successful
0811 12:06:08.028961 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 12:06:08.037117 425 raft_consensus.cc:1261] T c0d702d4922d4af4817459f7889e5d08 P 43d08b9a29cf42a48adb34153409186a [term 1 FOLLOWER]: Refusing update from remote peer 380d1e238e15456d98dee6b99cdf4f78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:06:08.037118 622 raft_consensus.cc:1261] T c0d702d4922d4af4817459f7889e5d08 P d7bf3363e68843e189b30a08dde3b52d [term 1 FOLLOWER]: Refusing update from remote peer 380d1e238e15456d98dee6b99cdf4f78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:06:08.038239 734 consensus_queue.cc:1035] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7bf3363e68843e189b30a08dde3b52d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 38705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:06:08.038554 737 consensus_queue.cc:1035] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43d08b9a29cf42a48adb34153409186a" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 44905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:06:08.050848 741 mvcc.cc:205] Tried to move back new op lower bound from 7317299069069737984 to 7317299068961394688. Current Snapshot: MvccSnapshot[applied={T|T < 7317299069069737984}]
I20260811 12:06:08.050923 743 mvcc.cc:205] Tried to move back new op lower bound from 7317299069069737984 to 7317299068961394688. Current Snapshot: MvccSnapshot[applied={T|T < 7317299069069737984}]
I20260811 12:06:08.066408 32447 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:55252:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:06:08.066702 32447 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:55252:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:06:08.070073 32447 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc: Sending DeleteTablet for 3 replicas of tablet c0d702d4922d4af4817459f7889e5d08
I20260811 12:06:08.071424 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 32512
I20260811 12:06:08.071415 32648 tablet_service.cc:1514] Processing DeleteTablet for tablet c0d702d4922d4af4817459f7889e5d08 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:06:08 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:42162
I20260811 12:06:08.071465 590 tablet_service.cc:1514] Processing DeleteTablet for tablet c0d702d4922d4af4817459f7889e5d08 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:06:08 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:45250
I20260811 12:06:08.072196 391 tablet_service.cc:1514] Processing DeleteTablet for tablet c0d702d4922d4af4817459f7889e5d08 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:06:08 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:46480
I20260811 12:06:08.077752 756 tablet_replica.cc:330] T c0d702d4922d4af4817459f7889e5d08 P 43d08b9a29cf42a48adb34153409186a: stopping tablet replica
I20260811 12:06:08.078368 756 raft_consensus.cc:2229] T c0d702d4922d4af4817459f7889e5d08 P 43d08b9a29cf42a48adb34153409186a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:06:08.079072 756 raft_consensus.cc:2258] T c0d702d4922d4af4817459f7889e5d08 P 43d08b9a29cf42a48adb34153409186a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:06:08.083357 756 ts_tablet_manager.cc:1905] T c0d702d4922d4af4817459f7889e5d08 P 43d08b9a29cf42a48adb34153409186a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:06:08.090222 754 tablet_replica.cc:330] T c0d702d4922d4af4817459f7889e5d08 P d7bf3363e68843e189b30a08dde3b52d: stopping tablet replica
I20260811 12:06:08.090842 754 raft_consensus.cc:2229] T c0d702d4922d4af4817459f7889e5d08 P d7bf3363e68843e189b30a08dde3b52d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:06:08.091499 754 raft_consensus.cc:2258] T c0d702d4922d4af4817459f7889e5d08 P d7bf3363e68843e189b30a08dde3b52d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:06:08.093849 715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:08.072359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42192 (local address 127.9.221.129:43519)
0811 12:06:08.072590 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:08.072599 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:08.072704 (+ 105us) server_negotiation.cc:407] Connection header received
0811 12:06:08.072906 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:08.072914 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:08.073016 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:08.073144 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:06:08.075323 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:08.076979 (+ 1656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:08.079518 (+ 2539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:08.080571 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:08.080740 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:08.084636 (+ 3896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:06:08.084652 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:06:08.084671 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:06:08.084726 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:06:08.089265 (+ 4539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:08.090333 (+ 1068us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:08.090340 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:08.090343 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:08.090557 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:08.091039 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:08.091044 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:08.091046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:08.091535 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:06:08.091839 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:08.093140 (+ 1301us) server_negotiation.cc:299] Negotiation successful
0811 12:06:08.093403 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 12:06:08.095212 32740 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:06:08.096097 756 ts_tablet_manager.cc:1918] T c0d702d4922d4af4817459f7889e5d08 P 43d08b9a29cf42a48adb34153409186a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:06:08.096338 756 log.cc:1196] T c0d702d4922d4af4817459f7889e5d08 P 43d08b9a29cf42a48adb34153409186a: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-1/wal/wals/c0d702d4922d4af4817459f7889e5d08
I20260811 12:06:08.096633 754 ts_tablet_manager.cc:1905] T c0d702d4922d4af4817459f7889e5d08 P d7bf3363e68843e189b30a08dde3b52d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:06:08.097038 756 ts_tablet_manager.cc:1939] T c0d702d4922d4af4817459f7889e5d08 P 43d08b9a29cf42a48adb34153409186a: Deleting consensus metadata
I20260811 12:06:08.099375 32433 catalog_manager.cc:4841] TS 43d08b9a29cf42a48adb34153409186a (127.9.221.130:44905): tablet c0d702d4922d4af4817459f7889e5d08 (table test-table [id=3e48426511814b3ca825422d0f4a2caf]) successfully deleted
I20260811 12:06:08.102988 754 ts_tablet_manager.cc:1918] T c0d702d4922d4af4817459f7889e5d08 P d7bf3363e68843e189b30a08dde3b52d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:06:08.103204 754 log.cc:1196] T c0d702d4922d4af4817459f7889e5d08 P d7bf3363e68843e189b30a08dde3b52d: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-2/wal/wals/c0d702d4922d4af4817459f7889e5d08
I20260811 12:06:08.103935 754 ts_tablet_manager.cc:1939] T c0d702d4922d4af4817459f7889e5d08 P d7bf3363e68843e189b30a08dde3b52d: Deleting consensus metadata
I20260811 12:06:08.105644 32430 catalog_manager.cc:4841] TS d7bf3363e68843e189b30a08dde3b52d (127.9.221.131:38705): tablet c0d702d4922d4af4817459f7889e5d08 (table test-table [id=3e48426511814b3ca825422d0f4a2caf]) successfully deleted
I20260811 12:06:08.197989 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 32512
I20260811 12:06:08.225538 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 32762
W20260811 12:06:08.225544 32433 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.221.129:43519 (error 108)
I20260811 12:06:08.227448 713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:08.227034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:43519 (local address 127.0.0.1:42208)
0811 12:06:08.227205 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:08.227306 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.129:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 12:06:08.227810 32433 catalog_manager.cc:4584] TS 380d1e238e15456d98dee6b99cdf4f78 (127.9.221.129:43519): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c0d702d4922d4af4817459f7889e5d08: Network error: Client connection negotiation failed: client connection to 127.9.221.129:43519: connect: Connection refused (error 111)
I20260811 12:06:08.240006 719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:08.226384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46492 (local address 127.9.221.130:44905)
0811 12:06:08.226603 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:08.226610 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:08.226631 (+ 21us) server_negotiation.cc:407] Connection header received
0811 12:06:08.226871 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:08.226877 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:08.226966 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:08.227077 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:06:08.228607 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:08.229410 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:08.230818 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:08.231298 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:08.231424 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:08.234402 (+ 2978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:06:08.234423 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:06:08.234438 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:06:08.234467 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:06:08.237160 (+ 2693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:08.237886 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:08.237895 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:08.237899 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:08.238023 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:08.238431 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:08.238437 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:08.238441 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:08.238821 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:06:08.239111 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:08.239422 (+ 311us) server_negotiation.cc:299] Negotiation successful
0811 12:06:08.239663 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 12:06:08.240818 488 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:06:08.281502 713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:08.280947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:43519 (local address 127.0.0.1:42212)
0811 12:06:08.281198 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:08.281306 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.129:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260811 12:06:08.282584 713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:08.282171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:43519 (local address 127.0.0.1:42214)
0811 12:06:08.282344 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:08.282442 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.129:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 12:06:08.331305 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 32762
I20260811 12:06:08.343250 713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:08.342725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:43519 (local address 127.0.0.1:42220)
0811 12:06:08.343004 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:08.343093 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.129:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140}
I20260811 12:06:08.344465 713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:08.344099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:43519 (local address 127.0.0.1:42236)
0811 12:06:08.344238 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:08.344304 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.129:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 12:06:08.358174 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 502
I20260811 12:06:08.371618 714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:08.358985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45266 (local address 127.9.221.131:38705)
0811 12:06:08.359177 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:08.359184 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:08.359206 (+ 22us) server_negotiation.cc:407] Connection header received
0811 12:06:08.359269 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:08.359275 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:08.359350 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:08.359470 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:06:08.360880 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:08.361949 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:08.363198 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:08.363582 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:08.363740 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:08.366348 (+ 2608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:06:08.366361 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:06:08.366377 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:06:08.366408 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:06:08.368993 (+ 2585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:08.369654 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:08.369660 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:08.369663 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:08.369794 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:08.370140 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:08.370143 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:08.370145 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:08.370470 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:06:08.370681 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:08.371084 (+ 403us) server_negotiation.cc:299] Negotiation successful
0811 12:06:08.371243 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 12:06:08.372291 692 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:06:08.425534 713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:08.424882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:43519 (local address 127.0.0.1:42250)
0811 12:06:08.425163 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:08.425300 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.129:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 12:06:08.426833 713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:08.426337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:43519 (local address 127.0.0.1:42264)
0811 12:06:08.426570 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:08.426672 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.129:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260811 12:06:08.477592 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 502
I20260811 12:06:08.502261 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 32404
I20260811 12:06:08.503501 32499 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:06:08.618525 713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:08.617823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:43519 (local address 127.0.0.1:42276)
0811 12:06:08.618112 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:08.618227 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.129:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 12:06:08.618872 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 32404
I20260811 12:06:08.651556 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:32893
--webserver_interface=127.9.221.190
--webserver_port=41771
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:37169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:32893
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:06:08.994195 763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:06:08.994477 763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:06:08.994679 763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:06:09.002079 763 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:06:09.002229 763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:06:09.002315 763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:06:09.002403 763 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:06:09.002468 763 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:06:09.012712 763 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:37169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:32893
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:32893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.221.190
--webserver_port=41771
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:06:09.014031 763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:06:09.015332 763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:06:09.027292 771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:06:09.027397 772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:06:09.027348 775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:06:09.028328 763 server_base.cc:1034] running on GCE node
I20260811 12:06:09.029112 763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:06:09.030464 763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:06:09.031734 763 hybrid_clock.cc:648] HybridClock initialized: now 1786449969031695 us; error 43 us; skew 500 ppm
I20260811 12:06:09.032078 763 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:06:09 dist-test-slave-mjjr krb5kdc[31590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449969, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:06:09.035223 763 init.cc:377] Logged in from keytab as kudu/127.9.221.190@KRBTEST.COM (short username kudu)
I20260811 12:06:09.036664 763 webserver.cc:460] Webserver started at http://127.9.221.190:41771/ using document root <none> and password file <none>
I20260811 12:06:09.037313 763 fs_manager.cc:359] Metadata directory not provided
I20260811 12:06:09.037462 763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:06:09.042572 763 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260811 12:06:09.045354 786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:06:09.046626 763 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 12:06:09.046804 763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "b26f2b577ada48839a0741a87dd73acc"
format_stamp: "Formatted at 2026-08-11 12:06:03 on dist-test-slave-mjjr"
server_key: "5a61fb635d34e6b1e61380c5b17ea827"
server_key_iv: "a76de3da28dd8119ae7c728dd9eec038"
server_key_version: "encryptionkey@0"
I20260811 12:06:09.047806 763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:06:09.074246 763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:06:09.078536 763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:06:09.078809 763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:06:09.097896 763 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.190:32893
I20260811 12:06:09.097915 864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.190:32893 every 8 connection(s)
I20260811 12:06:09.099483 763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:06:09.106211 866 sys_catalog.cc:263] Verifying existing consensus state
I20260811 12:06:09.109416 866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc: Bootstrap starting.
I20260811 12:06:09.109601 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 763
I20260811 12:06:09.110656 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.129:43519
--local_ip_for_outbound_sockets=127.9.221.129
--tserver_master_addrs=127.9.221.190:32893
--webserver_port=43271
--webserver_interface=127.9.221.129
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab
--principal=kudu/127.9.221.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:37169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:06:09.133879 866 log.cc:824] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc: Log is configured to *not* fsync() on all Append() calls
I20260811 12:06:09.149885 866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 12:06:09.150470 866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc: Bootstrap complete.
I20260811 12:06:09.159775 866 raft_consensus.cc:348] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b26f2b577ada48839a0741a87dd73acc" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 32893 } }
I20260811 12:06:09.160610 866 raft_consensus.cc:729] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: b26f2b577ada48839a0741a87dd73acc, State: Initialized, Role: FOLLOWER
I20260811 12:06:09.161226 866 consensus_queue.cc:260] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b26f2b577ada48839a0741a87dd73acc" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 32893 } }
I20260811 12:06:09.161438 866 raft_consensus.cc:388] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:06:09.161561 866 raft_consensus.cc:482] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:06:09.161746 866 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [term 2 FOLLOWER]: Advancing to term 3
I20260811 12:06:09.165643 866 raft_consensus.cc:504] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b26f2b577ada48839a0741a87dd73acc" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 32893 } }
I20260811 12:06:09.166127 866 leader_election.cc:302] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b26f2b577ada48839a0741a87dd73acc; no voters:
I20260811 12:06:09.166847 866 leader_election.cc:288] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 12:06:09.166993 871 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [term 3 FOLLOWER]: Leader election won for term 3
I20260811 12:06:09.168287 871 raft_consensus.cc:686] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [term 3 LEADER]: Becoming Leader. State: Replica: b26f2b577ada48839a0741a87dd73acc, State: Running, Role: LEADER
I20260811 12:06:09.168823 871 consensus_queue.cc:237] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b26f2b577ada48839a0741a87dd73acc" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 32893 } }
I20260811 12:06:09.169158 866 sys_catalog.cc:564] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:06:09.171331 874 sys_catalog.cc:455] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "b26f2b577ada48839a0741a87dd73acc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b26f2b577ada48839a0741a87dd73acc" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 32893 } } }
I20260811 12:06:09.171716 874 sys_catalog.cc:458] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [sys.catalog]: This master's current role is: LEADER
I20260811 12:06:09.171998 875 sys_catalog.cc:455] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [sys.catalog]: SysCatalogTable state changed. Reason: New leader b26f2b577ada48839a0741a87dd73acc. Latest consensus state: current_term: 3 leader_uuid: "b26f2b577ada48839a0741a87dd73acc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b26f2b577ada48839a0741a87dd73acc" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 32893 } } }
I20260811 12:06:09.172637 875 sys_catalog.cc:458] T 00000000000000000000000000000000 P b26f2b577ada48839a0741a87dd73acc [sys.catalog]: This master's current role is: LEADER
I20260811 12:06:09.180163 878 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:06:09.188687 878 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:06:09.190359 878 catalog_manager.cc:1259] Loaded cluster ID: 0441414dd2804a6984a06b101f730a8c
I20260811 12:06:09.190516 878 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260811 12:06:09.192865 878 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f1529f8f9ad google::LogMessage::Fail() at ??:0
@ 0x7f1529f9288a google::LogMessage::SendToLog() at ??:0
@ 0x7f1529f8f4de google::LogMessage::Flush() at ??:0
@ 0x7f1529f90a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f153f9c6512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f153f9c3976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
I20260811 12:06:09.246611 905 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7f153fa4b610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f153fa4b439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f152cbdb345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f152cbde91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f152cbde799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f152cbb9eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f152fe936db start_thread at ??:0
@ 0x7f1528e2961f clone at ??:0
W20260811 12:06:09.582145 868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:06:09.582419 868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:06:09.582583 868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:06:09.590010 868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:06:09.590122 868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:06:09.590272 868 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.129
I20260811 12:06:09.600241 868 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:37169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab
--principal=kudu/127.9.221.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.129:43519
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.221.129
--webserver_port=43271
--tserver_master_addrs=127.9.221.190:32893
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.129
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:06:09.601653 868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:06:09.603103 868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:06:09.616165 928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:06:09.616202 923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:06:09.616341 925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:06:09.617388 868 server_base.cc:1034] running on GCE node
I20260811 12:06:09.618045 868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:06:09.619055 868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:06:09.620296 868 hybrid_clock.cc:648] HybridClock initialized: now 1786449969620241 us; error 37 us; skew 500 ppm
I20260811 12:06:09.620630 868 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:06:09 dist-test-slave-mjjr krb5kdc[31590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449969, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:06:09.624053 868 init.cc:377] Logged in from keytab as kudu/127.9.221.129@KRBTEST.COM (short username kudu)
I20260811 12:06:09.625582 868 webserver.cc:460] Webserver started at http://127.9.221.129:43271/ using document root <none> and password file <none>
I20260811 12:06:09.626281 868 fs_manager.cc:359] Metadata directory not provided
I20260811 12:06:09.626441 868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:06:09.631949 868 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260811 12:06:09.636922 941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:06:09.638233 868 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:06:09.638414 868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-0/wal
uuid: "380d1e238e15456d98dee6b99cdf4f78"
format_stamp: "Formatted at 2026-08-11 12:06:03 on dist-test-slave-mjjr"
server_key: "aedf699819165298ab6f5baee3205207"
server_key_iv: "1ea069fc7f6394608d75adb016ee9668"
server_key_version: "encryptionkey@0"
I20260811 12:06:09.639411 868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:06:09.660406 868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:06:09.664774 868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:06:09.665061 868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:06:09.666038 868 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:06:09.667897 951 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260811 12:06:09.673427 868 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 12:06:09.673600 868 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260811 12:06:09.673753 868 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 12:06:09.676893 868 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 12:06:09.677007 868 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.000s sys 0.002s
I20260811 12:06:09.677326 951 tablet_bootstrap.cc:492] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78: Bootstrap starting.
I20260811 12:06:09.705241 951 log.cc:824] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78: Log is configured to *not* fsync() on all Append() calls
I20260811 12:06:09.723438 1124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.129:43519 every 8 connection(s)
I20260811 12:06:09.723446 868 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.129:43519
I20260811 12:06:09.725204 868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
I20260811 12:06:09.725600 951 tablet_bootstrap.cc:492] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 12:06:09.726500 951 tablet_bootstrap.cc:492] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78: Bootstrap complete.
I20260811 12:06:09.728482 951 ts_tablet_manager.cc:1397] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78: Time spent bootstrapping tablet: real 0.051s user 0.032s sys 0.016s
I20260811 12:06:09.730927 1132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:09.728383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.129:55661)
0811 12:06:09.729453 (+ 1070us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:09.729848 (+ 395us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":796,"thread_start_us":576,"threads_started":1}
W20260811 12:06:09.731695 1126 heartbeater.cc:644] Failed to heartbeat to 127.9.221.190:32893 (0 consecutive failures): Network error: Failed to ping master at 127.9.221.190:32893: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
I20260811 12:06:09.732738 1132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:09.732341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.129:55479)
0811 12:06:09.732557 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:09.732634 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260811 12:06:09.733655 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 868
I20260811 12:06:09.734066 1132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:09.733556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.129:41283)
0811 12:06:09.733746 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:09.733858 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260811 12:06:09.734396 1126 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 12:06:09.734570 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.130:44905
--local_ip_for_outbound_sockets=127.9.221.130
--tserver_master_addrs=127.9.221.190:32893
--webserver_port=41937
--webserver_interface=127.9.221.130
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab
--principal=kudu/127.9.221.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:37169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:06:09.736581 951 raft_consensus.cc:348] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7bf3363e68843e189b30a08dde3b52d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 38705 } } peers { permanent_uuid: "380d1e238e15456d98dee6b99cdf4f78" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43519 } } peers { permanent_uuid: "43d08b9a29cf42a48adb34153409186a" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 44905 } }
I20260811 12:06:09.737504 951 raft_consensus.cc:729] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 380d1e238e15456d98dee6b99cdf4f78, State: Initialized, Role: FOLLOWER
I20260811 12:06:09.738131 951 consensus_queue.cc:260] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7bf3363e68843e189b30a08dde3b52d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 38705 } } peers { permanent_uuid: "380d1e238e15456d98dee6b99cdf4f78" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43519 } } peers { permanent_uuid: "43d08b9a29cf42a48adb34153409186a" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 44905 } }
I20260811 12:06:09.739840 951 ts_tablet_manager.cc:1428] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78: Time spent starting tablet: real 0.011s user 0.011s sys 0.001s
I20260811 12:06:09.739956 1132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:09.739483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.129:49515)
0811 12:06:09.739691 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:09.739784 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 12:06:09.741014 1132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:09.740553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.129:49245)
0811 12:06:09.740762 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:09.740848 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260811 12:06:10.078521 1135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:06:10.078809 1135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:06:10.078902 1135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:06:10.085947 1135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:06:10.086063 1135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:06:10.086172 1135 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.130
I20260811 12:06:10.095821 1135 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:37169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab
--principal=kudu/127.9.221.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.130:44905
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.221.130
--webserver_port=41937
--tserver_master_addrs=127.9.221.190:32893
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.130
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:06:10.097093 1135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:06:10.098382 1135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:06:10.111507 1144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:06:10.111744 1146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:06:10.111824 1143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:06:10.112378 1135 server_base.cc:1034] running on GCE node
I20260811 12:06:10.113026 1135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:06:10.113924 1135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:06:10.115162 1135 hybrid_clock.cc:648] HybridClock initialized: now 1786449970115108 us; error 61 us; skew 500 ppm
I20260811 12:06:10.115481 1135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:06:10 dist-test-slave-mjjr krb5kdc[31590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449970, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:06:10.118718 1135 init.cc:377] Logged in from keytab as kudu/127.9.221.130@KRBTEST.COM (short username kudu)
I20260811 12:06:10.120263 1135 webserver.cc:460] Webserver started at http://127.9.221.130:41937/ using document root <none> and password file <none>
I20260811 12:06:10.120962 1135 fs_manager.cc:359] Metadata directory not provided
I20260811 12:06:10.121114 1135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:06:10.126519 1135 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 12:06:10.129269 1156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:06:10.130518 1135 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260811 12:06:10.130700 1135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-1/wal
uuid: "43d08b9a29cf42a48adb34153409186a"
format_stamp: "Formatted at 2026-08-11 12:06:04 on dist-test-slave-mjjr"
server_key: "f0f3ff1b9439b238d8bea92bddd09620"
server_key_iv: "ecca2f464b5cf5fc19d63d5ef807c261"
server_key_version: "encryptionkey@0"
I20260811 12:06:10.131846 1135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:06:10.164642 1135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:06:10.168720 1135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:06:10.169016 1135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:06:10.169953 1135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:06:10.171243 1135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:06:10.171377 1135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:06:10.171499 1135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:06:10.171612 1135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:06:10.205935 1135 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.130:44905
I20260811 12:06:10.206027 1332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.130:44905 every 8 connection(s)
I20260811 12:06:10.207425 1135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
I20260811 12:06:10.210273 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 1135
I20260811 12:06:10.211005 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.131:38705
--local_ip_for_outbound_sockets=127.9.221.131
--tserver_master_addrs=127.9.221.190:32893
--webserver_port=37811
--webserver_interface=127.9.221.131
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab
--principal=kudu/127.9.221.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:37169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:06:10.212848 1336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:10.210604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.130:53751)
0811 12:06:10.211627 (+ 1023us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:10.211974 (+ 347us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":736,"thread_start_us":361,"threads_started":1}
W20260811 12:06:10.213645 1334 heartbeater.cc:644] Failed to heartbeat to 127.9.221.190:32893 (0 consecutive failures): Network error: Failed to ping master at 127.9.221.190:32893: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
I20260811 12:06:10.214653 1336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:10.214286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.130:56293)
0811 12:06:10.214465 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:10.214546 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260811 12:06:10.215598 1336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:10.215283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.130:44535)
0811 12:06:10.215411 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:10.215488 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 12:06:10.215886 1334 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 12:06:10.543243 1337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:06:10.543567 1337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:06:10.543778 1337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:06:10.550978 1337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:06:10.551079 1337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:06:10.551239 1337 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.131
I20260811 12:06:10.561143 1337 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:37169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab
--principal=kudu/127.9.221.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.131:38705
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.221.131
--webserver_port=37811
--tserver_master_addrs=127.9.221.190:32893
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.131
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:06:10.562438 1337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:06:10.563803 1337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:06:10.576926 1345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:06:10.576968 1346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:06:10.577090 1349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:06:10.577878 1337 server_base.cc:1034] running on GCE node
I20260811 12:06:10.578456 1337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:06:10.579276 1337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:06:10.580521 1337 hybrid_clock.cc:648] HybridClock initialized: now 1786449970580463 us; error 30 us; skew 500 ppm
I20260811 12:06:10.580793 1337 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:06:10 dist-test-slave-mjjr krb5kdc[31590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449970, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:06:10.584040 1337 init.cc:377] Logged in from keytab as kudu/127.9.221.131@KRBTEST.COM (short username kudu)
I20260811 12:06:10.585412 1337 webserver.cc:460] Webserver started at http://127.9.221.131:37811/ using document root <none> and password file <none>
I20260811 12:06:10.586036 1337 fs_manager.cc:359] Metadata directory not provided
I20260811 12:06:10.586145 1337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:06:10.591603 1337 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 12:06:10.594388 1360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:06:10.595841 1337 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 12:06:10.595976 1337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-2/wal
uuid: "d7bf3363e68843e189b30a08dde3b52d"
format_stamp: "Formatted at 2026-08-11 12:06:04 on dist-test-slave-mjjr"
server_key: "5a0fef63527135fba81646238b899a8a"
server_key_iv: "4116aa837c816cac98532681b80e9e35"
server_key_version: "encryptionkey@0"
I20260811 12:06:10.596916 1337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:06:10.608042 1337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:06:10.612329 1337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:06:10.612541 1337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:06:10.613425 1337 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:06:10.614631 1337 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:06:10.614728 1337 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:06:10.614811 1337 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:06:10.614871 1337 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:06:10.649211 1337 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.131:38705
I20260811 12:06:10.649370 1521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.131:38705 every 8 connection(s)
I20260811 12:06:10.650676 1337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
I20260811 12:06:10.655395 1525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:10.653656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.131:43483)
0811 12:06:10.654356 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:10.654574 (+ 218us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":279,"threads_started":1}
W20260811 12:06:10.656302 1522 heartbeater.cc:644] Failed to heartbeat to 127.9.221.190:32893 (0 consecutive failures): Network error: Failed to ping master at 127.9.221.190:32893: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
I20260811 12:06:10.657081 1525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:10.656775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.131:56331)
0811 12:06:10.656904 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:10.656977 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 12:06:10.657877 1525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:10.657584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.131:36505)
0811 12:06:10.657705 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:10.657776 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 12:06:10.658118 1522 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 12:06:10.659578 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 1337
I20260811 12:06:10.742727 1527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:10.741907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.129:42821)
0811 12:06:10.742366 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:10.742458 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":241,"threads_started":1}
I20260811 12:06:11.217164 1528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:11.216359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.130:44721)
0811 12:06:11.216804 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:11.216897 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":222,"threads_started":1}
I20260811 12:06:11.558315 1531 raft_consensus.cc:482] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:06:11.558564 1531 raft_consensus.cc:504] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7bf3363e68843e189b30a08dde3b52d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 38705 } } peers { permanent_uuid: "380d1e238e15456d98dee6b99cdf4f78" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43519 } } peers { permanent_uuid: "43d08b9a29cf42a48adb34153409186a" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 44905 } }
I20260811 12:06:11.560349 1531 leader_election.cc:288] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d7bf3363e68843e189b30a08dde3b52d (127.9.221.131:38705), 43d08b9a29cf42a48adb34153409186a (127.9.221.130:44905)
Aug 11 12:06:11 dist-test-slave-mjjr krb5kdc[31590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449969, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for kudu/127.9.221.130@KRBTEST.COM
Aug 11 12:06:11 dist-test-slave-mjjr krb5kdc[31590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449969, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for kudu/127.9.221.131@KRBTEST.COM
I20260811 12:06:11.586563 1535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:11.560950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:47405 (local address 127.9.221.130:44905)
0811 12:06:11.562002 (+ 1052us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:11.562022 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:11.562616 (+ 594us) server_negotiation.cc:407] Connection header received
0811 12:06:11.564805 (+ 2189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:11.564813 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:11.565155 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:11.565484 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:06:11.567447 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:11.568629 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:11.570372 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:11.570983 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:11.571177 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:11.579340 (+ 8163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:06:11.579347 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:06:11.579360 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:06:11.579399 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:06:11.582262 (+ 2863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:11.583006 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:11.583020 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:11.583025 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:11.583169 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:11.583632 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:11.583664 (+ 32us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:11.583668 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:11.584346 (+ 678us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:06:11.584545 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:11.585741 (+ 1196us) server_negotiation.cc:299] Negotiation successful
0811 12:06:11.586127 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":836,"thread_start_us":653,"threads_started":1}
I20260811 12:06:11.586613 1536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:11.561005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:58523 (local address 127.9.221.131:38705)
0811 12:06:11.562596 (+ 1591us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:11.562615 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:11.563313 (+ 698us) server_negotiation.cc:407] Connection header received
0811 12:06:11.564913 (+ 1600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:11.564920 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:11.565245 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:11.565556 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:06:11.567435 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:11.568654 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:11.570376 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:11.570984 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:11.571188 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:11.579691 (+ 8503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:06:11.579702 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:06:11.579721 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:06:11.579758 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:06:11.582878 (+ 3120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:11.583546 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:11.583554 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:11.583586 (+ 32us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:11.583834 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:11.584287 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:11.584292 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:11.584296 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:11.584932 (+ 636us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:06:11.585116 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:11.585808 (+ 692us) server_negotiation.cc:299] Negotiation successful
0811 12:06:11.586169 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1409,"thread_start_us":1291,"threads_started":1}
I20260811 12:06:11.586618 1533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:11.560666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.130:44905 (local address 127.9.221.129:47405)
0811 12:06:11.562236 (+ 1570us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:11.562277 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 12:06:11.562948 (+ 671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:06:11.565604 (+ 2656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:06:11.565614 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:06:11.566574 (+ 960us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:06:11.567244 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:11.567262 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:11.568822 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:11.568826 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:11.570234 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:11.570246 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:11.571195 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:11.571202 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:11.571526 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:11.576577 (+ 5051us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:06:11.576608 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:06:11.579054 (+ 2446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:06:11.582410 (+ 3356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:06:11.582421 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:06:11.582438 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:06:11.582842 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:06:11.583298 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:06:11.583303 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:06:11.583306 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:06:11.583494 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:06:11.584635 (+ 1141us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:06:11.584643 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:06:11.584979 (+ 336us) client_negotiation.cc:769] Sending connection context
0811 12:06:11.585494 (+ 515us) client_negotiation.cc:240] Negotiation successful
0811 12:06:11.586090 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1449,"mutex_wait_us":152,"spinlock_wait_cycles":8448,"thread_start_us":1282,"threads_started":1}
I20260811 12:06:11.586719 1537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:11.560666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.131:38705 (local address 127.9.221.129:58523)
0811 12:06:11.563181 (+ 2515us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:11.563196 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:06:11.563356 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:06:11.565600 (+ 2244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:06:11.565611 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:06:11.566575 (+ 964us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:06:11.567251 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:11.567263 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:11.568834 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:11.568839 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:11.570235 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:11.570248 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:11.571262 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:11.571270 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:11.571500 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:11.576606 (+ 5106us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:06:11.576629 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:06:11.579368 (+ 2739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:06:11.583031 (+ 3663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:06:11.583038 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:06:11.583042 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:06:11.583409 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:06:11.583963 (+ 554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:06:11.583970 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:06:11.583973 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:06:11.584143 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:06:11.585077 (+ 934us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:06:11.585081 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:06:11.585222 (+ 141us) client_negotiation.cc:769] Sending connection context
0811 12:06:11.585543 (+ 321us) client_negotiation.cc:240] Negotiation successful
0811 12:06:11.586247 (+ 704us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2383,"mutex_wait_us":855,"spinlock_wait_cycles":31104,"thread_start_us":1639,"threads_started":1}
W20260811 12:06:11.590622 948 leader_election.cc:341] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 43d08b9a29cf42a48adb34153409186a (127.9.221.130:44905): Not found: Tablet not found: c0d702d4922d4af4817459f7889e5d08
W20260811 12:06:11.590921 947 leader_election.cc:341] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d7bf3363e68843e189b30a08dde3b52d (127.9.221.131:38705): Not found: Tablet not found: c0d702d4922d4af4817459f7889e5d08
I20260811 12:06:11.591048 947 leader_election.cc:302] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 380d1e238e15456d98dee6b99cdf4f78; no voters: 43d08b9a29cf42a48adb34153409186a, d7bf3363e68843e189b30a08dde3b52d
I20260811 12:06:11.591413 1531 raft_consensus.cc:2726] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 12:06:11.664599 1540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:11.662991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.131:46961)
0811 12:06:11.663787 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:11.664157 (+ 370us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":370,"threads_started":1}
I20260811 12:06:11.744247 1533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:11.743626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.129:33359)
0811 12:06:11.743897 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:11.744038 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260811 12:06:12.218976 1541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:12.218269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.130:42289)
0811 12:06:12.218716 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:12.218800 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":248,"threads_started":1}
I20260811 12:06:12.666091 1543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:12.665461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.131:39775)
0811 12:06:12.665863 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:12.665943 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":203,"threads_started":1}
I20260811 12:06:12.745936 1544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:12.745187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.129:33595)
0811 12:06:12.745678 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:12.745760 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":228,"threads_started":1}
I20260811 12:06:13.220603 1545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:13.219933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.130:43061)
0811 12:06:13.220363 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:13.220442 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":230,"threads_started":1}
I20260811 12:06:13.365913 1546 raft_consensus.cc:482] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:06:13.366147 1546 raft_consensus.cc:504] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7bf3363e68843e189b30a08dde3b52d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 38705 } } peers { permanent_uuid: "380d1e238e15456d98dee6b99cdf4f78" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43519 } } peers { permanent_uuid: "43d08b9a29cf42a48adb34153409186a" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 44905 } }
I20260811 12:06:13.366911 1546 leader_election.cc:288] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d7bf3363e68843e189b30a08dde3b52d (127.9.221.131:38705), 43d08b9a29cf42a48adb34153409186a (127.9.221.130:44905)
W20260811 12:06:13.367924 947 leader_election.cc:341] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d7bf3363e68843e189b30a08dde3b52d (127.9.221.131:38705): Not found: Tablet not found: c0d702d4922d4af4817459f7889e5d08
W20260811 12:06:13.368079 948 leader_election.cc:341] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 43d08b9a29cf42a48adb34153409186a (127.9.221.130:44905): Not found: Tablet not found: c0d702d4922d4af4817459f7889e5d08
I20260811 12:06:13.368283 948 leader_election.cc:302] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 380d1e238e15456d98dee6b99cdf4f78; no voters: 43d08b9a29cf42a48adb34153409186a, d7bf3363e68843e189b30a08dde3b52d
I20260811 12:06:13.368553 1546 raft_consensus.cc:2726] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 12:06:13.667683 1547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:13.666962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.131:36357)
0811 12:06:13.667393 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:13.667475 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":223,"threads_started":1}
I20260811 12:06:13.747562 1548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:13.746796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.129:52583)
0811 12:06:13.747242 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:13.747362 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":227,"threads_started":1}
I20260811 12:06:14.222251 1549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:14.221472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.130:53167)
0811 12:06:14.221989 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:14.222081 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":295,"threads_started":1}
I20260811 12:06:14.669267 1550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:14.668572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.131:41277)
0811 12:06:14.669009 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:14.669088 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":231,"threads_started":1}
I20260811 12:06:14.749243 1552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:14.748587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.129:52777)
0811 12:06:14.748984 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:14.749067 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":215,"threads_started":1}
I20260811 12:06:15.224223 1553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:15.223182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.130:43243)
0811 12:06:15.223883 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:15.223992 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":389,"threads_started":1}
I20260811 12:06:15.671144 1554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:15.670354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.131:60997)
0811 12:06:15.670857 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:15.670970 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":279,"threads_started":1}
I20260811 12:06:15.750912 1556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:15.750208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.129:33531)
0811 12:06:15.750667 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:15.750748 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":258,"threads_started":1}
I20260811 12:06:15.912528 1557 raft_consensus.cc:482] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:06:15.912792 1557 raft_consensus.cc:504] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7bf3363e68843e189b30a08dde3b52d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 38705 } } peers { permanent_uuid: "380d1e238e15456d98dee6b99cdf4f78" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43519 } } peers { permanent_uuid: "43d08b9a29cf42a48adb34153409186a" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 44905 } }
I20260811 12:06:15.913554 1557 leader_election.cc:288] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d7bf3363e68843e189b30a08dde3b52d (127.9.221.131:38705), 43d08b9a29cf42a48adb34153409186a (127.9.221.130:44905)
W20260811 12:06:15.914579 948 leader_election.cc:341] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 43d08b9a29cf42a48adb34153409186a (127.9.221.130:44905): Not found: Tablet not found: c0d702d4922d4af4817459f7889e5d08
W20260811 12:06:15.914793 947 leader_election.cc:341] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d7bf3363e68843e189b30a08dde3b52d (127.9.221.131:38705): Not found: Tablet not found: c0d702d4922d4af4817459f7889e5d08
I20260811 12:06:15.914933 947 leader_election.cc:302] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 380d1e238e15456d98dee6b99cdf4f78; no voters: 43d08b9a29cf42a48adb34153409186a, d7bf3363e68843e189b30a08dde3b52d
I20260811 12:06:15.915153 1557 raft_consensus.cc:2726] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 12:06:16.225785 1558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:16.225067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.130:46499)
0811 12:06:16.225513 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:16.225601 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":238,"threads_started":1}
I20260811 12:06:16.672840 1560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:16.672084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.131:45863)
0811 12:06:16.672593 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:16.672673 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":280,"threads_started":1}
I20260811 12:06:16.752593 1561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:16.751847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.129:36807)
0811 12:06:16.752290 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:16.752390 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":243,"threads_started":1}
I20260811 12:06:17.227715 1562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:17.226748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.130:36211)
0811 12:06:17.227377 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:17.227470 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":325,"threads_started":1}
I20260811 12:06:17.674715 1564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:17.673871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.131:49101)
0811 12:06:17.674428 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:17.674530 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":293,"threads_started":1}
I20260811 12:06:17.754251 1565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:17.753567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.129:33831)
0811 12:06:17.754008 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:17.754094 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":230,"threads_started":1}
I20260811 12:06:18.229291 1567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:18.228591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.130:46431)
0811 12:06:18.229021 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:18.229112 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":210,"threads_started":1}
I20260811 12:06:18.676688 1568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:18.675898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.131:51251)
0811 12:06:18.676397 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:18.676488 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":264,"threads_started":1}
I20260811 12:06:18.756062 1569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:18.755224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.129:34753)
0811 12:06:18.755758 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:18.755864 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":246,"threads_started":1}
I20260811 12:06:19.231009 1570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:19.230286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.130:53475)
0811 12:06:19.230756 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:19.230840 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":243,"threads_started":1}
I20260811 12:06:19.678815 1571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:19.677838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.131:45111)
0811 12:06:19.678577 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:19.678658 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":639,"thread_start_us":392,"threads_started":1}
I20260811 12:06:19.757838 1572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:19.757085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.129:59169)
0811 12:06:19.757545 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:19.757633 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":246,"threads_started":1}
I20260811 12:06:20.031252 1573 raft_consensus.cc:482] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:06:20.031527 1573 raft_consensus.cc:504] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7bf3363e68843e189b30a08dde3b52d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 38705 } } peers { permanent_uuid: "380d1e238e15456d98dee6b99cdf4f78" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43519 } } peers { permanent_uuid: "43d08b9a29cf42a48adb34153409186a" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 44905 } }
I20260811 12:06:20.032361 1573 leader_election.cc:288] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d7bf3363e68843e189b30a08dde3b52d (127.9.221.131:38705), 43d08b9a29cf42a48adb34153409186a (127.9.221.130:44905)
W20260811 12:06:20.033387 948 leader_election.cc:341] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 43d08b9a29cf42a48adb34153409186a (127.9.221.130:44905): Not found: Tablet not found: c0d702d4922d4af4817459f7889e5d08
W20260811 12:06:20.033587 947 leader_election.cc:341] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d7bf3363e68843e189b30a08dde3b52d (127.9.221.131:38705): Not found: Tablet not found: c0d702d4922d4af4817459f7889e5d08
I20260811 12:06:20.033715 947 leader_election.cc:302] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 380d1e238e15456d98dee6b99cdf4f78; no voters: 43d08b9a29cf42a48adb34153409186a, d7bf3363e68843e189b30a08dde3b52d
I20260811 12:06:20.033995 1573 raft_consensus.cc:2726] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 12:06:20.232690 1574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:20.231913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.130:50063)
0811 12:06:20.232336 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:20.232457 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":221,"threads_started":1}
I20260811 12:06:20.680776 1576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:20.679940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.131:56543)
0811 12:06:20.680520 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:20.680600 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":310,"threads_started":1}
I20260811 12:06:20.759614 1578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:20.758875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.129:52695)
0811 12:06:20.759349 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:20.759438 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":255,"threads_started":1}
I20260811 12:06:21.234299 1579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:21.233583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.130:56367)
0811 12:06:21.234040 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:21.234130 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":238,"threads_started":1}
I20260811 12:06:21.682871 1580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:21.682126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.131:37333)
0811 12:06:21.682609 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:21.682712 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":239,"threads_started":1}
I20260811 12:06:21.761626 1581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:21.760818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.129:38041)
0811 12:06:21.761320 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:21.761418 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":253,"threads_started":1}
I20260811 12:06:22.236203 1583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:22.235359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.130:37041)
0811 12:06:22.235939 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:22.236030 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":296,"threads_started":1}
I20260811 12:06:22.684664 1584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:22.683977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.131:60977)
0811 12:06:22.684404 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:22.684494 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":237,"threads_started":1}
I20260811 12:06:22.763386 1585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:22.762652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.129:41425)
0811 12:06:22.763127 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:22.763211 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":239,"threads_started":1}
I20260811 12:06:23.058215 1587 raft_consensus.cc:482] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:06:23.058491 1587 raft_consensus.cc:504] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7bf3363e68843e189b30a08dde3b52d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 38705 } } peers { permanent_uuid: "380d1e238e15456d98dee6b99cdf4f78" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43519 } } peers { permanent_uuid: "43d08b9a29cf42a48adb34153409186a" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 44905 } }
I20260811 12:06:23.059221 1587 leader_election.cc:288] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d7bf3363e68843e189b30a08dde3b52d (127.9.221.131:38705), 43d08b9a29cf42a48adb34153409186a (127.9.221.130:44905)
W20260811 12:06:23.060202 947 leader_election.cc:341] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d7bf3363e68843e189b30a08dde3b52d (127.9.221.131:38705): Not found: Tablet not found: c0d702d4922d4af4817459f7889e5d08
W20260811 12:06:23.060371 948 leader_election.cc:341] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 43d08b9a29cf42a48adb34153409186a (127.9.221.130:44905): Not found: Tablet not found: c0d702d4922d4af4817459f7889e5d08
I20260811 12:06:23.060534 948 leader_election.cc:302] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 380d1e238e15456d98dee6b99cdf4f78; no voters: 43d08b9a29cf42a48adb34153409186a, d7bf3363e68843e189b30a08dde3b52d
I20260811 12:06:23.060767 1587 raft_consensus.cc:2726] T c0d702d4922d4af4817459f7889e5d08 P 380d1e238e15456d98dee6b99cdf4f78 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 12:06:23.237771 1589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:23.237060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.130:35171)
0811 12:06:23.237503 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:23.237595 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":219,"threads_started":1}
I20260811 12:06:23.686385 1591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:23.685672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.131:35261)
0811 12:06:23.686131 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:23.686218 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":270,"threads_started":1}
I20260811 12:06:23.765055 1592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:23.764348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.129:39601)
0811 12:06:23.764784 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:23.764867 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":218,"threads_started":1}
I20260811 12:06:24.239380 1597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:24.238657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.130:40163)
0811 12:06:24.239114 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:24.239194 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":240,"threads_started":1}
I20260811 12:06:24.688131 1599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:24.687297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.131:46929)
0811 12:06:24.687839 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:24.687922 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":260,"threads_started":1}
I20260811 12:06:24.766976 1600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:24.766249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.129:54713)
0811 12:06:24.766715 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:24.766796 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":264,"threads_started":1}
I20260811 12:06:25.241137 1602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:25.240408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.130:45465)
0811 12:06:25.240876 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:25.240967 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":236,"threads_started":1}
I20260811 12:06:25.668614 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 868
I20260811 12:06:25.688957 1604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:25.669836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49180 (local address 127.9.221.129:43519)
0811 12:06:25.670365 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:25.670373 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:25.670400 (+ 27us) server_negotiation.cc:407] Connection header received
0811 12:06:25.670473 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:25.670479 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:25.670743 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:25.670880 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:06:25.672852 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:25.673796 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:25.674978 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:25.675545 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:25.675694 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:25.683552 (+ 7858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:06:25.683564 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:06:25.683578 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:06:25.683625 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:06:25.686397 (+ 2772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:25.687054 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:25.687060 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:25.687062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:25.687181 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:25.687514 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:25.687517 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:25.687519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:25.687910 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:06:25.688167 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:25.688424 (+ 257us) server_negotiation.cc:299] Negotiation successful
0811 12:06:25.688610 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":245,"threads_started":1}
I20260811 12:06:25.689981 1606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:25.689265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:32893 (local address 127.9.221.131:35545)
0811 12:06:25.689757 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:25.689831 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":269,"threads_started":1}
I20260811 12:06:25.690378 1118 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:06:25.780360 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 868
I20260811 12:06:25.809226 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 1135
I20260811 12:06:25.823105 1609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:25.809987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54492 (local address 127.9.221.130:44905)
0811 12:06:25.810513 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:25.810517 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:25.810532 (+ 15us) server_negotiation.cc:407] Connection header received
0811 12:06:25.810587 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:25.810592 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:25.810649 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:25.810777 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:06:25.812309 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:25.813106 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:25.814354 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:25.814786 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:25.814933 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:25.817702 (+ 2769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:06:25.817707 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:06:25.817710 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:06:25.817742 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:06:25.820322 (+ 2580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:25.821011 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:25.821018 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:25.821021 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:25.821177 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:25.821602 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:25.821607 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:25.821610 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:25.821877 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:06:25.822175 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:25.822490 (+ 315us) server_negotiation.cc:299] Negotiation successful
0811 12:06:25.822715 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":287,"threads_started":1}
I20260811 12:06:25.824030 1326 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:06:25.922454 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 1135
I20260811 12:06:25.951506 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 1337
I20260811 12:06:25.964764 1613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:25.952404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51892 (local address 127.9.221.131:38705)
0811 12:06:25.952923 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:25.952935 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:25.952952 (+ 17us) server_negotiation.cc:407] Connection header received
0811 12:06:25.953007 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:25.953011 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:25.953070 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:25.953160 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:06:25.954701 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:25.955468 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:25.956760 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:25.957194 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:25.957290 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:25.959849 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:06:25.959854 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:06:25.959857 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:06:25.959888 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:06:25.962369 (+ 2481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:25.963017 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:25.963021 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:25.963023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:25.963161 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:25.963471 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:25.963474 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:25.963476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:25.963783 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:06:25.963991 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:25.964213 (+ 222us) server_negotiation.cc:299] Negotiation successful
0811 12:06:25.964369 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":257,"threads_started":1}
I20260811 12:06:25.965459 1515 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:06:26.049437 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 1337
2026-08-11T12:06:26Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24780 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:06:26 dist-test-slave-mjjr krb5kdc[1620](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:06:26 dist-test-slave-mjjr krb5kdc[1620](info): set up 2 sockets
Aug 11 12:06:26 dist-test-slave-mjjr krb5kdc[1620](info): commencing operation
krb5kdc: starting...
W20260811 12:06:27.376545 10102 mini_kdc.cc:121] Time spent starting KDC: real 1.284s user 0.003s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Aug 11 12:06:27 dist-test-slave-mjjr krb5kdc[1620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:06:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:06:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.221.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
I20260811 12:06:27.650343 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:34171
--webserver_interface=127.9.221.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:43273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:34171
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:06:27.991166 1646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:06:27.991537 1646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:06:27.991700 1646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:06:27.998950 1646 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 12:06:27.999075 1646 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 12:06:27.999127 1646 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 12:06:27.999174 1646 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:06:27.999228 1646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:06:27.999296 1646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:06:27.999342 1646 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:06:27.999390 1646 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:06:28.009295 1646 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:43273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:34171
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:34171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.221.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:06:28.010491 1646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:06:28.011829 1646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:06:28.024044 1652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:06:28.024005 1653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:06:28.024083 1656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:06:28.024346 1646 server_base.cc:1034] running on GCE node
I20260811 12:06:28.025316 1646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:06:28.026638 1646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:06:28.027889 1646 hybrid_clock.cc:648] HybridClock initialized: now 1786449988027826 us; error 36 us; skew 500 ppm
I20260811 12:06:28.028224 1646 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:06:28 dist-test-slave-mjjr krb5kdc[1620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449988, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:06:28.031234 1646 init.cc:377] Logged in from keytab as kudu/127.9.221.190@KRBTEST.COM (short username kudu)
I20260811 12:06:28.032675 1646 webserver.cc:460] Webserver started at http://127.9.221.190:33093/ using document root <none> and password file <none>
I20260811 12:06:28.033608 1646 fs_manager.cc:359] Metadata directory not provided
I20260811 12:06:28.033757 1646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:06:28.034077 1646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:06:28.037113 1646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "c4c1846038424feca3e6885cfad8f589"
format_stamp: "Formatted at 2026-08-11 12:06:28 on dist-test-slave-mjjr"
server_key: "ecfb6a6f8a3f136d21e129edb48df75b"
server_key_iv: "55aafc83132132325cfde0295656c37a"
server_key_version: "encryptionkey@0"
I20260811 12:06:28.037900 1646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "c4c1846038424feca3e6885cfad8f589"
format_stamp: "Formatted at 2026-08-11 12:06:28 on dist-test-slave-mjjr"
server_key: "ecfb6a6f8a3f136d21e129edb48df75b"
server_key_iv: "55aafc83132132325cfde0295656c37a"
server_key_version: "encryptionkey@0"
I20260811 12:06:28.042909 1646 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260811 12:06:28.046478 1664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:06:28.047760 1646 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:06:28.047935 1646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "c4c1846038424feca3e6885cfad8f589"
format_stamp: "Formatted at 2026-08-11 12:06:28 on dist-test-slave-mjjr"
server_key: "ecfb6a6f8a3f136d21e129edb48df75b"
server_key_iv: "55aafc83132132325cfde0295656c37a"
server_key_version: "encryptionkey@0"
I20260811 12:06:28.048141 1646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:06:28.066361 1646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:06:28.070156 1646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:06:28.070432 1646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:06:28.089208 1646 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.190:34171
I20260811 12:06:28.089216 1738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.190:34171 every 8 connection(s)
I20260811 12:06:28.090533 1646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:06:28.094271 1740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:06:28.096144 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 1646
I20260811 12:06:28.096382 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:06:28.097218 10102 external_mini_cluster.cc:1428] Setting key c6d14045a01539470bcb03c79ea7dd71
I20260811 12:06:28.108927 1740 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589: Bootstrap starting.
Aug 11 12:06:28 dist-test-slave-mjjr krb5kdc[1620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:06:28.112932 1740 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589: Neither blocks nor log segments found. Creating new log.
I20260811 12:06:28.114032 1740 log.cc:824] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589: Log is configured to *not* fsync() on all Append() calls
I20260811 12:06:28.117120 1740 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589: No bootstrap required, opened a new log
I20260811 12:06:28.121028 1744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:28.099358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50870 (local address 127.9.221.190:34171)
0811 12:06:28.100230 (+ 872us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:28.100240 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:28.100286 (+ 46us) server_negotiation.cc:407] Connection header received
0811 12:06:28.101806 (+ 1520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:28.101829 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:28.102184 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:28.102471 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:06:28.104351 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:28.105652 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:28.107610 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:28.108304 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:28.108431 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:28.113482 (+ 5051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:06:28.113490 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:06:28.113503 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:06:28.113540 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:06:28.116858 (+ 3318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:28.117665 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:28.117671 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:28.117677 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:28.117842 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:28.118241 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:28.118246 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:28.118250 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:28.118767 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:06:28.119097 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:28.119578 (+ 481us) server_negotiation.cc:299] Negotiation successful
0811 12:06:28.120037 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":669,"thread_start_us":451,"threads_started":1}
I20260811 12:06:28.125141 1740 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4c1846038424feca3e6885cfad8f589" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 34171 } }
I20260811 12:06:28.125525 1740 raft_consensus.cc:374] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:06:28.125638 1740 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4c1846038424feca3e6885cfad8f589, State: Initialized, Role: FOLLOWER
I20260811 12:06:28.126168 1740 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4c1846038424feca3e6885cfad8f589" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 34171 } }
I20260811 12:06:28.126422 1740 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:06:28.126595 1740 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:06:28.126775 1740 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:06:28.129170 1740 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4c1846038424feca3e6885cfad8f589" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 34171 } }
I20260811 12:06:28.129745 1740 leader_election.cc:302] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4c1846038424feca3e6885cfad8f589; no voters:
I20260811 12:06:28.130378 1740 leader_election.cc:288] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:06:28.130493 1746 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:06:28.131263 1746 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [term 1 LEADER]: Becoming Leader. State: Replica: c4c1846038424feca3e6885cfad8f589, State: Running, Role: LEADER
I20260811 12:06:28.132030 1746 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4c1846038424feca3e6885cfad8f589" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 34171 } }
I20260811 12:06:28.132700 1740 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:06:28.134696 1747 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c4c1846038424feca3e6885cfad8f589" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4c1846038424feca3e6885cfad8f589" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 34171 } } }
I20260811 12:06:28.135041 1747 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [sys.catalog]: This master's current role is: LEADER
I20260811 12:06:28.135941 1752 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:06:28.135784 1748 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4c1846038424feca3e6885cfad8f589. Latest consensus state: current_term: 1 leader_uuid: "c4c1846038424feca3e6885cfad8f589" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4c1846038424feca3e6885cfad8f589" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 34171 } } }
I20260811 12:06:28.136168 1748 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [sys.catalog]: This master's current role is: LEADER
I20260811 12:06:28.141287 1752 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:06:28.150424 1752 catalog_manager.cc:1347] Generated new cluster ID: 7ce147b8a8934af5a9e5329269f93ede
I20260811 12:06:28.150560 1752 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:06:28.159619 1752 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:06:28.160842 1752 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:06:28.178318 1752 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589: Generated new TSK 0
I20260811 12:06:28.179167 1752 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.221.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab.
Entry for principal kudu/127.9.221.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab.
I20260811 12:06:28.251065 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.129:0
--local_ip_for_outbound_sockets=127.9.221.129
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:34171
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab
--principal=kudu/127.9.221.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:43273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:06:28.608676 1778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:06:28.608954 1778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:06:28.609047 1778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:06:28.616080 1778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:06:28.616179 1778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:06:28.616287 1778 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.129
I20260811 12:06:28.625790 1778 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:43273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab
--principal=kudu/127.9.221.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:34171
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.129
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:06:28.627030 1778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:06:28.628484 1778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:06:28.641798 1786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:06:28.641880 1791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:06:28.641866 1787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:06:28.642769 1778 server_base.cc:1034] running on GCE node
I20260811 12:06:28.643327 1778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:06:28.644256 1778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:06:28.645488 1778 hybrid_clock.cc:648] HybridClock initialized: now 1786449988645451 us; error 28 us; skew 500 ppm
I20260811 12:06:28.645792 1778 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:06:28 dist-test-slave-mjjr krb5kdc[1620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449988, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:06:28.649009 1778 init.cc:377] Logged in from keytab as kudu/127.9.221.129@KRBTEST.COM (short username kudu)
I20260811 12:06:28.650312 1778 webserver.cc:460] Webserver started at http://127.9.221.129:46257/ using document root <none> and password file <none>
I20260811 12:06:28.650914 1778 fs_manager.cc:359] Metadata directory not provided
I20260811 12:06:28.651021 1778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:06:28.651270 1778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:06:28.654247 1778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-0/data/instance:
uuid: "54580f2d27444280a5f21772db02ac05"
format_stamp: "Formatted at 2026-08-11 12:06:28 on dist-test-slave-mjjr"
server_key: "d3da32d3f13ebd7b30eaa096fa5738a8"
server_key_iv: "e9ff98d4c1ae70a2f0f0c454955b1a6a"
server_key_version: "encryptionkey@0"
I20260811 12:06:28.654928 1778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance:
uuid: "54580f2d27444280a5f21772db02ac05"
format_stamp: "Formatted at 2026-08-11 12:06:28 on dist-test-slave-mjjr"
server_key: "d3da32d3f13ebd7b30eaa096fa5738a8"
server_key_iv: "e9ff98d4c1ae70a2f0f0c454955b1a6a"
server_key_version: "encryptionkey@0"
I20260811 12:06:28.659849 1778 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260811 12:06:28.663156 1801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:06:28.664413 1778 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:06:28.664597 1778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-0/wal
uuid: "54580f2d27444280a5f21772db02ac05"
format_stamp: "Formatted at 2026-08-11 12:06:28 on dist-test-slave-mjjr"
server_key: "d3da32d3f13ebd7b30eaa096fa5738a8"
server_key_iv: "e9ff98d4c1ae70a2f0f0c454955b1a6a"
server_key_version: "encryptionkey@0"
I20260811 12:06:28.664789 1778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:06:28.680064 1778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:06:28.684361 1778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:06:28.684657 1778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:06:28.685599 1778 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:06:28.686899 1778 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:06:28.686976 1778 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:06:28.687140 1778 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:06:28.687199 1778 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:06:28.720404 1778 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.129:45207
I20260811 12:06:28.720461 1968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.129:45207 every 8 connection(s)
I20260811 12:06:28.721762 1778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
I20260811 12:06:28.729979 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 1778
I20260811 12:06:28.730211 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance
I20260811 12:06:28.730906 10102 external_mini_cluster.cc:1428] Setting key f9f018f9db1497511ac08abcd07d1282
Aug 11 12:06:28 dist-test-slave-mjjr krb5kdc[1620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449988, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:06:28.742565 1972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:28.724449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:39881 (local address 127.9.221.190:34171)
0811 12:06:28.724924 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:28.724935 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:28.725798 (+ 863us) server_negotiation.cc:407] Connection header received
0811 12:06:28.727601 (+ 1803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:28.727606 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:28.727715 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:28.727839 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:06:28.729318 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:28.730129 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:28.731848 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:28.732283 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:28.732458 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:28.736765 (+ 4307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:06:28.736770 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:06:28.736773 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:06:28.736806 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:06:28.739350 (+ 2544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:28.740203 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:28.740214 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:28.740217 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:28.740336 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:28.740782 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:28.740787 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:28.740790 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:28.741064 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:06:28.741223 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:28.741956 (+ 733us) server_negotiation.cc:299] Negotiation successful
0811 12:06:28.742189 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":263,"threads_started":1}
I20260811 12:06:28.743527 1973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:28.724911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.129:39881)
0811 12:06:28.725638 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:28.725675 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 12:06:28.727321 (+ 1646us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:06:28.728083 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:06:28.728091 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:06:28.728608 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:06:28.729146 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:28.729163 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:28.730300 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:28.730305 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:28.731718 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:28.731737 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:28.732604 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:28.732614 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:28.732887 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:28.733642 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:06:28.733674 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:06:28.736539 (+ 2865us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:06:28.739502 (+ 2963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:06:28.739511 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:06:28.739528 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:06:28.740069 (+ 541us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:06:28.740456 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:06:28.740462 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:06:28.740465 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:06:28.740660 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:06:28.741196 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:06:28.741204 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:06:28.741570 (+ 366us) client_negotiation.cc:769] Sending connection context
0811 12:06:28.741832 (+ 262us) client_negotiation.cc:240] Negotiation successful
0811 12:06:28.742370 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":273,"threads_started":1}
I20260811 12:06:28.745302 1969 heartbeater.cc:347] Connected to a master server at 127.9.221.190:34171
I20260811 12:06:28.745635 1969 heartbeater.cc:464] Registering TS with master...
I20260811 12:06:28.746542 1969 heartbeater.cc:511] Master 127.9.221.190:34171 requested a full tablet report, sending...
I20260811 12:06:28.748708 1686 ts_manager.cc:194] Registered new tserver with Master: 54580f2d27444280a5f21772db02ac05 (127.9.221.129:45207)
I20260811 12:06:28.750135 1686 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.129@KRBTEST.COM'} at 127.9.221.129:39881
WARNING: no policy specified for kudu/127.9.221.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab.
Entry for principal kudu/127.9.221.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab.
I20260811 12:06:28.795408 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.130:0
--local_ip_for_outbound_sockets=127.9.221.130
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:34171
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab
--principal=kudu/127.9.221.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:43273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:06:29.131402 1980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:06:29.131690 1980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:06:29.131788 1980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:06:29.139196 1980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:06:29.139295 1980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:06:29.139446 1980 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.130
I20260811 12:06:29.149230 1980 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:43273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab
--principal=kudu/127.9.221.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:34171
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.130
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:06:29.150496 1980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:06:29.152037 1980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:06:29.165360 1986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:06:29.165385 1991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:06:29.165485 1989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:06:29.166507 1980 server_base.cc:1034] running on GCE node
I20260811 12:06:29.167047 1980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:06:29.168089 1980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:06:29.169329 1980 hybrid_clock.cc:648] HybridClock initialized: now 1786449989169264 us; error 48 us; skew 500 ppm
I20260811 12:06:29.169591 1980 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:06:29 dist-test-slave-mjjr krb5kdc[1620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449989, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:06:29.172876 1980 init.cc:377] Logged in from keytab as kudu/127.9.221.130@KRBTEST.COM (short username kudu)
I20260811 12:06:29.174355 1980 webserver.cc:460] Webserver started at http://127.9.221.130:42069/ using document root <none> and password file <none>
I20260811 12:06:29.175010 1980 fs_manager.cc:359] Metadata directory not provided
I20260811 12:06:29.175146 1980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:06:29.175493 1980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:06:29.178443 1980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-1/data/instance:
uuid: "225c9ecec8ac4f2ca6aa89c9f5d8fc03"
format_stamp: "Formatted at 2026-08-11 12:06:29 on dist-test-slave-mjjr"
server_key: "6f941d3e7dcecd5716e78cc05912f543"
server_key_iv: "adf3c9a4373acd93250cde0880f14d5d"
server_key_version: "encryptionkey@0"
I20260811 12:06:29.179315 1980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance:
uuid: "225c9ecec8ac4f2ca6aa89c9f5d8fc03"
format_stamp: "Formatted at 2026-08-11 12:06:29 on dist-test-slave-mjjr"
server_key: "6f941d3e7dcecd5716e78cc05912f543"
server_key_iv: "adf3c9a4373acd93250cde0880f14d5d"
server_key_version: "encryptionkey@0"
I20260811 12:06:29.184991 1980 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260811 12:06:29.188421 2001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:06:29.189632 1980 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260811 12:06:29.189811 1980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-1/wal
uuid: "225c9ecec8ac4f2ca6aa89c9f5d8fc03"
format_stamp: "Formatted at 2026-08-11 12:06:29 on dist-test-slave-mjjr"
server_key: "6f941d3e7dcecd5716e78cc05912f543"
server_key_iv: "adf3c9a4373acd93250cde0880f14d5d"
server_key_version: "encryptionkey@0"
I20260811 12:06:29.190021 1980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:06:29.208387 1980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:06:29.212172 1980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:06:29.212484 1980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:06:29.213380 1980 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:06:29.214699 1980 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:06:29.214809 1980 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:06:29.214952 1980 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:06:29.215029 1980 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:06:29.249012 1980 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.130:35285
I20260811 12:06:29.249068 2162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.130:35285 every 8 connection(s)
I20260811 12:06:29.250341 1980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
I20260811 12:06:29.253121 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 1980
I20260811 12:06:29.253334 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance
I20260811 12:06:29.253922 10102 external_mini_cluster.cc:1428] Setting key 45be371457e4e77d3ccda6ea7338df69
Aug 11 12:06:29 dist-test-slave-mjjr krb5kdc[1620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449989, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.130@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:06:29.272224 2165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:29.253000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:55949 (local address 127.9.221.190:34171)
0811 12:06:29.253527 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:29.253532 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:29.254630 (+ 1098us) server_negotiation.cc:407] Connection header received
0811 12:06:29.257019 (+ 2389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:29.257025 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:29.257112 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:29.257229 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:06:29.258690 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:29.259410 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:29.261038 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:29.261447 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:29.261593 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:29.265471 (+ 3878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:06:29.265479 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:06:29.265487 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:06:29.265532 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:06:29.268860 (+ 3328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:29.269843 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:29.269847 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:29.269849 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:29.269972 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:29.270455 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:29.270458 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:29.270460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:29.270708 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:06:29.270814 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:29.271799 (+ 985us) server_negotiation.cc:299] Negotiation successful
0811 12:06:29.271964 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":301,"threads_started":1}
I20260811 12:06:29.273739 2166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:29.253687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.130:55949)
0811 12:06:29.254448 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:29.254494 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 12:06:29.256748 (+ 2254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:06:29.257459 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:06:29.257467 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:06:29.257985 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:06:29.258521 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:29.258538 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:29.259590 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:29.259594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:29.260879 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:29.260892 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:29.261685 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:29.261692 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:29.261909 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:29.262741 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:06:29.262777 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:06:29.265233 (+ 2456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:06:29.269000 (+ 3767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:06:29.269011 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:06:29.269028 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:06:29.269723 (+ 695us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:06:29.270083 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:06:29.270091 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:06:29.270097 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:06:29.270340 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:06:29.270836 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:06:29.270846 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:06:29.271350 (+ 504us) client_negotiation.cc:769] Sending connection context
0811 12:06:29.271692 (+ 342us) client_negotiation.cc:240] Negotiation successful
0811 12:06:29.272256 (+ 564us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":329,"threads_started":1}
I20260811 12:06:29.275218 2163 heartbeater.cc:347] Connected to a master server at 127.9.221.190:34171
I20260811 12:06:29.275568 2163 heartbeater.cc:464] Registering TS with master...
I20260811 12:06:29.276535 2163 heartbeater.cc:511] Master 127.9.221.190:34171 requested a full tablet report, sending...
I20260811 12:06:29.278199 1686 ts_manager.cc:194] Registered new tserver with Master: 225c9ecec8ac4f2ca6aa89c9f5d8fc03 (127.9.221.130:35285)
I20260811 12:06:29.279031 1686 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.130@KRBTEST.COM'} at 127.9.221.130:55949
WARNING: no policy specified for kudu/127.9.221.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab.
Entry for principal kudu/127.9.221.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab.
I20260811 12:06:29.318553 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.131:0
--local_ip_for_outbound_sockets=127.9.221.131
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:34171
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab
--principal=kudu/127.9.221.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:43273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:06:29.653738 2170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:06:29.654035 2170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:06:29.654208 2170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:06:29.661386 2170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:06:29.661527 2170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:06:29.661700 2170 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.131
I20260811 12:06:29.671272 2170 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:43273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab
--principal=kudu/127.9.221.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:34171
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.131
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:06:29.672549 2170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:06:29.673904 2170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:06:29.687058 2177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:06:29.687009 2178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:06:29.687093 2180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:06:29.687865 2170 server_base.cc:1034] running on GCE node
I20260811 12:06:29.688382 2170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:06:29.689232 2170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:06:29.690441 2170 hybrid_clock.cc:648] HybridClock initialized: now 1786449989690403 us; error 37 us; skew 500 ppm
I20260811 12:06:29.690702 2170 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:06:29 dist-test-slave-mjjr krb5kdc[1620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449989, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:06:29.693845 2170 init.cc:377] Logged in from keytab as kudu/127.9.221.131@KRBTEST.COM (short username kudu)
I20260811 12:06:29.695348 2170 webserver.cc:460] Webserver started at http://127.9.221.131:45199/ using document root <none> and password file <none>
I20260811 12:06:29.696202 2170 fs_manager.cc:359] Metadata directory not provided
I20260811 12:06:29.696345 2170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:06:29.696666 2170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:06:29.699540 2170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-2/data/instance:
uuid: "9c5816077f504b37892ccc03751f3120"
format_stamp: "Formatted at 2026-08-11 12:06:29 on dist-test-slave-mjjr"
server_key: "b3c89d8bf9581a010408e82c2c3d518b"
server_key_iv: "1c565945cf5ad91d331d3c59ca71a56e"
server_key_version: "encryptionkey@0"
I20260811 12:06:29.700418 2170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance:
uuid: "9c5816077f504b37892ccc03751f3120"
format_stamp: "Formatted at 2026-08-11 12:06:29 on dist-test-slave-mjjr"
server_key: "b3c89d8bf9581a010408e82c2c3d518b"
server_key_iv: "1c565945cf5ad91d331d3c59ca71a56e"
server_key_version: "encryptionkey@0"
I20260811 12:06:29.705410 2170 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:06:29.708699 2189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:06:29.709931 2170 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 12:06:29.710100 2170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-2/wal
uuid: "9c5816077f504b37892ccc03751f3120"
format_stamp: "Formatted at 2026-08-11 12:06:29 on dist-test-slave-mjjr"
server_key: "b3c89d8bf9581a010408e82c2c3d518b"
server_key_iv: "1c565945cf5ad91d331d3c59ca71a56e"
server_key_version: "encryptionkey@0"
I20260811 12:06:29.710291 2170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:06:29.736438 2170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:06:29.740821 2170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:06:29.741175 2170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:06:29.742206 2170 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:06:29.743594 2170 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:06:29.743747 2170 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260811 12:06:29.743862 2170 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:06:29.743916 2170 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:06:29.753831 1969 heartbeater.cc:503] Master 127.9.221.190:34171 was elected leader, sending a full tablet report...
I20260811 12:06:29.779052 2170 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.131:41145
I20260811 12:06:29.779095 2361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.131:41145 every 8 connection(s)
I20260811 12:06:29.780359 2170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
I20260811 12:06:29.785898 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 2170
I20260811 12:06:29.786099 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance
I20260811 12:06:29.786592 10102 external_mini_cluster.cc:1428] Setting key 99e2b7a1d372302b2e22c20606177ba1
Aug 11 12:06:29 dist-test-slave-mjjr krb5kdc[1620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449989, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.131@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:06:29.800307 2366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:29.782959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:51133 (local address 127.9.221.190:34171)
0811 12:06:29.783430 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:29.783434 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:29.784435 (+ 1001us) server_negotiation.cc:407] Connection header received
0811 12:06:29.786296 (+ 1861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:29.786303 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:29.786390 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:29.786532 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:06:29.788260 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:29.789419 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:29.790942 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:29.791328 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:29.791455 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:29.794956 (+ 3501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:06:29.794960 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:06:29.794964 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:06:29.794996 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:06:29.797587 (+ 2591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:29.798234 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:29.798239 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:29.798242 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:29.798361 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:29.798717 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:29.798720 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:29.798722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:29.798970 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:06:29.799090 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:29.799736 (+ 646us) server_negotiation.cc:299] Negotiation successful
0811 12:06:29.799933 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":235,"threads_started":1}
I20260811 12:06:29.801414 2368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:29.783506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.131:51133)
0811 12:06:29.784258 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:29.784302 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 12:06:29.785956 (+ 1654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:06:29.786776 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:06:29.786784 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:06:29.787360 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:06:29.788088 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:29.788114 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:29.789573 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:29.789579 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:29.790783 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:29.790799 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:29.791557 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:29.791564 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:29.791838 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:29.792471 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:06:29.792492 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:06:29.794773 (+ 2281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:06:29.797710 (+ 2937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:06:29.797716 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:06:29.797726 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:06:29.798125 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:06:29.798451 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:06:29.798454 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:06:29.798456 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:06:29.798619 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:06:29.799071 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:06:29.799089 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:06:29.799348 (+ 259us) client_negotiation.cc:769] Sending connection context
0811 12:06:29.799550 (+ 202us) client_negotiation.cc:240] Negotiation successful
0811 12:06:29.800110 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":329,"threads_started":1}
I20260811 12:06:29.802752 2362 heartbeater.cc:347] Connected to a master server at 127.9.221.190:34171
I20260811 12:06:29.803009 2362 heartbeater.cc:464] Registering TS with master...
I20260811 12:06:29.803624 2362 heartbeater.cc:511] Master 127.9.221.190:34171 requested a full tablet report, sending...
I20260811 12:06:29.805011 1686 ts_manager.cc:194] Registered new tserver with Master: 9c5816077f504b37892ccc03751f3120 (127.9.221.131:41145)
I20260811 12:06:29.805811 1686 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.131@KRBTEST.COM'} at 127.9.221.131:51133
I20260811 12:06:29.814105 10102 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:06:29.831936 2366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:29.818673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50876 (local address 127.9.221.190:34171)
0811 12:06:29.818904 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:29.818912 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:29.819286 (+ 374us) server_negotiation.cc:407] Connection header received
0811 12:06:29.819485 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:29.819490 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:29.819560 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:29.819695 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:06:29.821180 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:29.821960 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:29.823353 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:29.823855 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:29.823993 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:29.826641 (+ 2648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:06:29.826647 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:06:29.826650 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:06:29.826683 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:06:29.829265 (+ 2582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:29.829920 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:29.829927 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:29.829930 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:29.830061 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:29.830452 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:29.830458 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:29.830461 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:29.830789 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:06:29.831063 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:29.831375 (+ 312us) server_negotiation.cc:299] Negotiation successful
0811 12:06:29.831598 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 12:06:29.835853 1686 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50876:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:06:29.843019 1686 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:06:29.880771 2386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:29.854819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47060 (local address 127.9.221.131:41145)
0811 12:06:29.869215 (+ 14396us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:29.869224 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:29.869262 (+ 38us) server_negotiation.cc:407] Connection header received
0811 12:06:29.869676 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:29.869684 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:29.871057 (+ 1373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:29.873016 (+ 1959us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:06:29.873747 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:29.875546 (+ 1799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:29.878104 (+ 2558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:29.879578 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:29.879761 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:29.880000 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:29.880238 (+ 238us) server_negotiation.cc:299] Negotiation successful
0811 12:06:29.880409 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14023,"thread_start_us":13598,"threads_started":1}
I20260811 12:06:29.881613 2387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:29.854507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.131:41145 (local address 127.0.0.1:47060)
0811 12:06:29.866308 (+ 11801us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:29.866354 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 12:06:29.867027 (+ 673us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:06:29.872351 (+ 5324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:06:29.872358 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:06:29.872391 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:06:29.873511 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:29.873528 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:29.875732 (+ 2204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:29.875739 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:29.877906 (+ 2167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:29.877923 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:29.879798 (+ 1875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:29.879805 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:29.879988 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:29.880039 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 12:06:29.880173 (+ 134us) client_negotiation.cc:240] Negotiation successful
0811 12:06:29.880351 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10952,"thread_start_us":18993,"threads_started":1}
I20260811 12:06:29.882216 2390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:29.855834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41010 (local address 127.9.221.129:45207)
0811 12:06:29.872699 (+ 16865us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:29.872710 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:29.874016 (+ 1306us) server_negotiation.cc:407] Connection header received
0811 12:06:29.874118 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:29.874126 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:29.874372 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:29.874570 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:06:29.875572 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:29.877186 (+ 1614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:29.879265 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:29.880564 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:29.880704 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:29.880952 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:29.881726 (+ 774us) server_negotiation.cc:299] Negotiation successful
0811 12:06:29.881890 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16638,"thread_start_us":16467,"threads_started":1}
I20260811 12:06:29.882277 2388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:29.855164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.130:35285 (local address 127.0.0.1:46296)
0811 12:06:29.872978 (+ 17814us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:29.872998 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:06:29.873123 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:06:29.874756 (+ 1633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:06:29.874760 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:06:29.874788 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:06:29.875335 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:29.875353 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:29.877465 (+ 2112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:29.877471 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:29.879613 (+ 2142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:29.879629 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:29.881439 (+ 1810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:29.881447 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:29.881625 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:29.881675 (+ 50us) client_negotiation.cc:769] Sending connection context
0811 12:06:29.881747 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 12:06:29.881869 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17679,"mutex_wait_us":5,"thread_start_us":9800,"threads_started":1}
I20260811 12:06:29.882674 2385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:29.855494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:45207 (local address 127.0.0.1:41010)
0811 12:06:29.873736 (+ 18242us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:29.873755 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:06:29.873883 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:06:29.874819 (+ 936us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:06:29.874825 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:06:29.874850 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:06:29.875370 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:29.875383 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:29.877370 (+ 1987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:29.877375 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:29.879099 (+ 1724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:29.879110 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:29.881089 (+ 1979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:29.881095 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:29.881255 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:29.881284 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 12:06:29.881371 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 12:06:29.881501 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18010,"mutex_wait_us":103,"spinlock_wait_cycles":2304,"thread_start_us":17225,"threads_started":1}
I20260811 12:06:29.883241 2389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:29.855702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46296 (local address 127.9.221.130:35285)
0811 12:06:29.874197 (+ 18495us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:29.874205 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:29.874230 (+ 25us) server_negotiation.cc:407] Connection header received
0811 12:06:29.874315 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:29.874323 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:29.874526 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:29.875571 (+ 1045us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:06:29.875862 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:29.877303 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:29.879804 (+ 2501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:29.881189 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:29.882593 (+ 1404us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:29.882755 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:29.882826 (+ 71us) server_negotiation.cc:299] Negotiation successful
0811 12:06:29.882981 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18326,"thread_start_us":19261,"threads_started":1}
I20260811 12:06:29.885857 1871 tablet_service.cc:1467] Processing CreateTablet for tablet 70838fcabe564bd39ba20a5dfd5aa188 (DEFAULT_TABLE table=test-table [id=890e715fbcb34e1d8478da3b73a8d678]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:06:29.886019 2272 tablet_service.cc:1467] Processing CreateTablet for tablet 70838fcabe564bd39ba20a5dfd5aa188 (DEFAULT_TABLE table=test-table [id=890e715fbcb34e1d8478da3b73a8d678]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:06:29.886457 2068 tablet_service.cc:1467] Processing CreateTablet for tablet 70838fcabe564bd39ba20a5dfd5aa188 (DEFAULT_TABLE table=test-table [id=890e715fbcb34e1d8478da3b73a8d678]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:06:29.887223 1871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70838fcabe564bd39ba20a5dfd5aa188. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:06:29.887480 2272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70838fcabe564bd39ba20a5dfd5aa188. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:06:29.887784 2068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70838fcabe564bd39ba20a5dfd5aa188. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:06:29.898970 2392 tablet_bootstrap.cc:492] T 70838fcabe564bd39ba20a5dfd5aa188 P 54580f2d27444280a5f21772db02ac05: Bootstrap starting.
I20260811 12:06:29.901744 2393 tablet_bootstrap.cc:492] T 70838fcabe564bd39ba20a5dfd5aa188 P 9c5816077f504b37892ccc03751f3120: Bootstrap starting.
I20260811 12:06:29.902424 2392 tablet_bootstrap.cc:654] T 70838fcabe564bd39ba20a5dfd5aa188 P 54580f2d27444280a5f21772db02ac05: Neither blocks nor log segments found. Creating new log.
I20260811 12:06:29.903810 2392 log.cc:824] T 70838fcabe564bd39ba20a5dfd5aa188 P 54580f2d27444280a5f21772db02ac05: Log is configured to *not* fsync() on all Append() calls
I20260811 12:06:29.904315 2394 tablet_bootstrap.cc:492] T 70838fcabe564bd39ba20a5dfd5aa188 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03: Bootstrap starting.
I20260811 12:06:29.905289 2393 tablet_bootstrap.cc:654] T 70838fcabe564bd39ba20a5dfd5aa188 P 9c5816077f504b37892ccc03751f3120: Neither blocks nor log segments found. Creating new log.
I20260811 12:06:29.906778 2393 log.cc:824] T 70838fcabe564bd39ba20a5dfd5aa188 P 9c5816077f504b37892ccc03751f3120: Log is configured to *not* fsync() on all Append() calls
I20260811 12:06:29.906927 2394 tablet_bootstrap.cc:654] T 70838fcabe564bd39ba20a5dfd5aa188 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03: Neither blocks nor log segments found. Creating new log.
I20260811 12:06:29.908176 2394 log.cc:824] T 70838fcabe564bd39ba20a5dfd5aa188 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03: Log is configured to *not* fsync() on all Append() calls
I20260811 12:06:29.909941 2392 tablet_bootstrap.cc:492] T 70838fcabe564bd39ba20a5dfd5aa188 P 54580f2d27444280a5f21772db02ac05: No bootstrap required, opened a new log
I20260811 12:06:29.910200 2392 ts_tablet_manager.cc:1397] T 70838fcabe564bd39ba20a5dfd5aa188 P 54580f2d27444280a5f21772db02ac05: Time spent bootstrapping tablet: real 0.011s user 0.010s sys 0.000s
I20260811 12:06:29.911084 2393 tablet_bootstrap.cc:492] T 70838fcabe564bd39ba20a5dfd5aa188 P 9c5816077f504b37892ccc03751f3120: No bootstrap required, opened a new log
I20260811 12:06:29.911336 2393 ts_tablet_manager.cc:1397] T 70838fcabe564bd39ba20a5dfd5aa188 P 9c5816077f504b37892ccc03751f3120: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260811 12:06:29.912103 2394 tablet_bootstrap.cc:492] T 70838fcabe564bd39ba20a5dfd5aa188 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03: No bootstrap required, opened a new log
I20260811 12:06:29.912355 2394 ts_tablet_manager.cc:1397] T 70838fcabe564bd39ba20a5dfd5aa188 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260811 12:06:29.918972 2393 raft_consensus.cc:348] T 70838fcabe564bd39ba20a5dfd5aa188 P 9c5816077f504b37892ccc03751f3120 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c5816077f504b37892ccc03751f3120" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41145 } } peers { permanent_uuid: "54580f2d27444280a5f21772db02ac05" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 45207 } } peers { permanent_uuid: "225c9ecec8ac4f2ca6aa89c9f5d8fc03" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 35285 } }
I20260811 12:06:29.919329 2393 raft_consensus.cc:374] T 70838fcabe564bd39ba20a5dfd5aa188 P 9c5816077f504b37892ccc03751f3120 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:06:29.919450 2393 raft_consensus.cc:729] T 70838fcabe564bd39ba20a5dfd5aa188 P 9c5816077f504b37892ccc03751f3120 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c5816077f504b37892ccc03751f3120, State: Initialized, Role: FOLLOWER
I20260811 12:06:29.920104 2393 consensus_queue.cc:260] T 70838fcabe564bd39ba20a5dfd5aa188 P 9c5816077f504b37892ccc03751f3120 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c5816077f504b37892ccc03751f3120" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41145 } } peers { permanent_uuid: "54580f2d27444280a5f21772db02ac05" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 45207 } } peers { permanent_uuid: "225c9ecec8ac4f2ca6aa89c9f5d8fc03" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 35285 } }
I20260811 12:06:29.920730 2392 raft_consensus.cc:348] T 70838fcabe564bd39ba20a5dfd5aa188 P 54580f2d27444280a5f21772db02ac05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c5816077f504b37892ccc03751f3120" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41145 } } peers { permanent_uuid: "54580f2d27444280a5f21772db02ac05" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 45207 } } peers { permanent_uuid: "225c9ecec8ac4f2ca6aa89c9f5d8fc03" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 35285 } }
I20260811 12:06:29.921126 2392 raft_consensus.cc:374] T 70838fcabe564bd39ba20a5dfd5aa188 P 54580f2d27444280a5f21772db02ac05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:06:29.921213 2392 raft_consensus.cc:729] T 70838fcabe564bd39ba20a5dfd5aa188 P 54580f2d27444280a5f21772db02ac05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54580f2d27444280a5f21772db02ac05, State: Initialized, Role: FOLLOWER
I20260811 12:06:29.921746 2392 consensus_queue.cc:260] T 70838fcabe564bd39ba20a5dfd5aa188 P 54580f2d27444280a5f21772db02ac05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c5816077f504b37892ccc03751f3120" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41145 } } peers { permanent_uuid: "54580f2d27444280a5f21772db02ac05" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 45207 } } peers { permanent_uuid: "225c9ecec8ac4f2ca6aa89c9f5d8fc03" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 35285 } }
I20260811 12:06:29.923028 2362 heartbeater.cc:503] Master 127.9.221.190:34171 was elected leader, sending a full tablet report...
I20260811 12:06:29.923543 2393 ts_tablet_manager.cc:1428] T 70838fcabe564bd39ba20a5dfd5aa188 P 9c5816077f504b37892ccc03751f3120: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260811 12:06:29.922771 2394 raft_consensus.cc:348] T 70838fcabe564bd39ba20a5dfd5aa188 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c5816077f504b37892ccc03751f3120" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41145 } } peers { permanent_uuid: "54580f2d27444280a5f21772db02ac05" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 45207 } } peers { permanent_uuid: "225c9ecec8ac4f2ca6aa89c9f5d8fc03" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 35285 } }
I20260811 12:06:29.925056 2394 raft_consensus.cc:374] T 70838fcabe564bd39ba20a5dfd5aa188 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:06:29.925160 2394 raft_consensus.cc:729] T 70838fcabe564bd39ba20a5dfd5aa188 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 225c9ecec8ac4f2ca6aa89c9f5d8fc03, State: Initialized, Role: FOLLOWER
I20260811 12:06:29.925697 2394 consensus_queue.cc:260] T 70838fcabe564bd39ba20a5dfd5aa188 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c5816077f504b37892ccc03751f3120" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41145 } } peers { permanent_uuid: "54580f2d27444280a5f21772db02ac05" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 45207 } } peers { permanent_uuid: "225c9ecec8ac4f2ca6aa89c9f5d8fc03" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 35285 } }
I20260811 12:06:29.927461 2163 heartbeater.cc:503] Master 127.9.221.190:34171 was elected leader, sending a full tablet report...
I20260811 12:06:29.927472 2392 ts_tablet_manager.cc:1428] T 70838fcabe564bd39ba20a5dfd5aa188 P 54580f2d27444280a5f21772db02ac05: Time spent starting tablet: real 0.017s user 0.003s sys 0.013s
I20260811 12:06:29.928260 2394 ts_tablet_manager.cc:1428] T 70838fcabe564bd39ba20a5dfd5aa188 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03: Time spent starting tablet: real 0.016s user 0.014s sys 0.000s
W20260811 12:06:29.974088 1971 tablet.cc:2369] T 70838fcabe564bd39ba20a5dfd5aa188 P 54580f2d27444280a5f21772db02ac05: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:06:29.982209 2401 raft_consensus.cc:482] T 70838fcabe564bd39ba20a5dfd5aa188 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:06:29.982546 2401 raft_consensus.cc:504] T 70838fcabe564bd39ba20a5dfd5aa188 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c5816077f504b37892ccc03751f3120" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41145 } } peers { permanent_uuid: "54580f2d27444280a5f21772db02ac05" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 45207 } } peers { permanent_uuid: "225c9ecec8ac4f2ca6aa89c9f5d8fc03" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 35285 } }
I20260811 12:06:29.984364 2401 leader_election.cc:288] T 70838fcabe564bd39ba20a5dfd5aa188 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c5816077f504b37892ccc03751f3120 (127.9.221.131:41145), 54580f2d27444280a5f21772db02ac05 (127.9.221.129:45207)
I20260811 12:06:29.991470 2390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:29.984985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:47083 (local address 127.9.221.129:45207)
0811 12:06:29.985245 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:29.985253 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:29.985891 (+ 638us) server_negotiation.cc:407] Connection header received
0811 12:06:29.986154 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:29.986158 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:29.986219 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:29.986317 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:06:29.986981 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:29.987775 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:29.989654 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:29.990433 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:29.990615 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:29.990704 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:29.991040 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 12:06:29.991169 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 12:06:29.991508 2386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:29.984848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:46679 (local address 127.9.221.131:41145)
0811 12:06:29.985089 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:29.985098 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:29.985872 (+ 774us) server_negotiation.cc:407] Connection header received
0811 12:06:29.986098 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:29.986102 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:29.986165 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:29.986267 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:06:29.986917 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:29.987764 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:29.989668 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:29.990440 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:29.990636 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:29.990720 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:29.991055 (+ 335us) server_negotiation.cc:299] Negotiation successful
0811 12:06:29.991179 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 12:06:29.991634 2402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:29.984548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.131:41145 (local address 127.9.221.130:46679)
0811 12:06:29.985730 (+ 1182us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:29.985767 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 12:06:29.985942 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:06:29.986285 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:06:29.986289 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:06:29.986313 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:06:29.986756 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:29.986768 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:29.987890 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:29.987897 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:29.989509 (+ 1612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:29.989522 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:29.990709 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:29.990715 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:29.990893 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:29.990915 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 12:06:29.991019 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 12:06:29.991155 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1007,"mutex_wait_us":88,"thread_start_us":772,"threads_started":1}
I20260811 12:06:29.992043 2403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:29.984800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:45207 (local address 127.9.221.130:47083)
0811 12:06:29.985782 (+ 982us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:29.985798 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:06:29.986004 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:06:29.986332 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:06:29.986336 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:06:29.986358 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:06:29.986810 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:29.986820 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:29.987906 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:29.987911 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:29.989499 (+ 1588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:29.989511 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:29.990675 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:29.990681 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:29.990875 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:29.990903 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 12:06:29.991016 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 12:06:29.991168 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":819,"mutex_wait_us":96,"spinlock_wait_cycles":26880,"thread_start_us":627,"threads_started":1}
I20260811 12:06:29.992741 2301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70838fcabe564bd39ba20a5dfd5aa188" candidate_uuid: "225c9ecec8ac4f2ca6aa89c9f5d8fc03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c5816077f504b37892ccc03751f3120" is_pre_election: true
I20260811 12:06:29.992997 1900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70838fcabe564bd39ba20a5dfd5aa188" candidate_uuid: "225c9ecec8ac4f2ca6aa89c9f5d8fc03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54580f2d27444280a5f21772db02ac05" is_pre_election: true
I20260811 12:06:29.993269 2301 raft_consensus.cc:2454] T 70838fcabe564bd39ba20a5dfd5aa188 P 9c5816077f504b37892ccc03751f3120 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 225c9ecec8ac4f2ca6aa89c9f5d8fc03 in term 0.
I20260811 12:06:29.993568 1900 raft_consensus.cc:2454] T 70838fcabe564bd39ba20a5dfd5aa188 P 54580f2d27444280a5f21772db02ac05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 225c9ecec8ac4f2ca6aa89c9f5d8fc03 in term 0.
I20260811 12:06:29.994016 2006 leader_election.cc:302] T 70838fcabe564bd39ba20a5dfd5aa188 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 225c9ecec8ac4f2ca6aa89c9f5d8fc03, 9c5816077f504b37892ccc03751f3120; no voters:
I20260811 12:06:29.994436 2401 raft_consensus.cc:2781] T 70838fcabe564bd39ba20a5dfd5aa188 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:06:29.994585 2401 raft_consensus.cc:482] T 70838fcabe564bd39ba20a5dfd5aa188 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:06:29.994685 2401 raft_consensus.cc:3037] T 70838fcabe564bd39ba20a5dfd5aa188 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:06:29.996949 2401 raft_consensus.cc:504] T 70838fcabe564bd39ba20a5dfd5aa188 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c5816077f504b37892ccc03751f3120" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41145 } } peers { permanent_uuid: "54580f2d27444280a5f21772db02ac05" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 45207 } } peers { permanent_uuid: "225c9ecec8ac4f2ca6aa89c9f5d8fc03" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 35285 } }
I20260811 12:06:29.997577 2401 leader_election.cc:288] T 70838fcabe564bd39ba20a5dfd5aa188 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03 [CANDIDATE]: Term 1 election: Requested vote from peers 9c5816077f504b37892ccc03751f3120 (127.9.221.131:41145), 54580f2d27444280a5f21772db02ac05 (127.9.221.129:45207)
I20260811 12:06:29.998000 2301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70838fcabe564bd39ba20a5dfd5aa188" candidate_uuid: "225c9ecec8ac4f2ca6aa89c9f5d8fc03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c5816077f504b37892ccc03751f3120"
I20260811 12:06:29.998104 1900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70838fcabe564bd39ba20a5dfd5aa188" candidate_uuid: "225c9ecec8ac4f2ca6aa89c9f5d8fc03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54580f2d27444280a5f21772db02ac05"
I20260811 12:06:29.998224 2301 raft_consensus.cc:3037] T 70838fcabe564bd39ba20a5dfd5aa188 P 9c5816077f504b37892ccc03751f3120 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:06:29.998296 1900 raft_consensus.cc:3037] T 70838fcabe564bd39ba20a5dfd5aa188 P 54580f2d27444280a5f21772db02ac05 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:06:30.001514 2301 raft_consensus.cc:2454] T 70838fcabe564bd39ba20a5dfd5aa188 P 9c5816077f504b37892ccc03751f3120 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 225c9ecec8ac4f2ca6aa89c9f5d8fc03 in term 1.
I20260811 12:06:30.001502 1900 raft_consensus.cc:2454] T 70838fcabe564bd39ba20a5dfd5aa188 P 54580f2d27444280a5f21772db02ac05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 225c9ecec8ac4f2ca6aa89c9f5d8fc03 in term 1.
I20260811 12:06:30.001950 2006 leader_election.cc:302] T 70838fcabe564bd39ba20a5dfd5aa188 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 225c9ecec8ac4f2ca6aa89c9f5d8fc03, 9c5816077f504b37892ccc03751f3120; no voters:
W20260811 12:06:30.002338 2164 tablet.cc:2369] T 70838fcabe564bd39ba20a5dfd5aa188 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:06:30.003078 2401 raft_consensus.cc:2781] T 70838fcabe564bd39ba20a5dfd5aa188 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:06:30.003511 2401 raft_consensus.cc:686] T 70838fcabe564bd39ba20a5dfd5aa188 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03 [term 1 LEADER]: Becoming Leader. State: Replica: 225c9ecec8ac4f2ca6aa89c9f5d8fc03, State: Running, Role: LEADER
I20260811 12:06:30.004040 2401 consensus_queue.cc:237] T 70838fcabe564bd39ba20a5dfd5aa188 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c5816077f504b37892ccc03751f3120" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41145 } } peers { permanent_uuid: "54580f2d27444280a5f21772db02ac05" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 45207 } } peers { permanent_uuid: "225c9ecec8ac4f2ca6aa89c9f5d8fc03" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 35285 } }
I20260811 12:06:30.008387 1686 catalog_manager.cc:5495] T 70838fcabe564bd39ba20a5dfd5aa188 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03 reported cstate change: term changed from 0 to 1, leader changed from <none> to 225c9ecec8ac4f2ca6aa89c9f5d8fc03 (127.9.221.130). New cstate: current_term: 1 leader_uuid: "225c9ecec8ac4f2ca6aa89c9f5d8fc03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c5816077f504b37892ccc03751f3120" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54580f2d27444280a5f21772db02ac05" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 45207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "225c9ecec8ac4f2ca6aa89c9f5d8fc03" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 35285 } health_report { overall_health: HEALTHY } } }
I20260811 12:06:30.023294 2389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:30.018210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46298 (local address 127.9.221.130:35285)
0811 12:06:30.018379 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:30.018384 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:30.018517 (+ 133us) server_negotiation.cc:407] Connection header received
0811 12:06:30.018679 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:30.018683 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:30.018770 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:30.018890 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:06:30.019232 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:30.019987 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:30.021336 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:30.021817 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:30.021951 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:30.022376 (+ 425us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:06:30.022832 (+ 456us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:06:30.022942 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:30.023032 (+ 90us) server_negotiation.cc:299] Negotiation successful
0811 12:06:30.023094 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 12:06:30.030900 2301 raft_consensus.cc:1261] T 70838fcabe564bd39ba20a5dfd5aa188 P 9c5816077f504b37892ccc03751f3120 [term 1 FOLLOWER]: Refusing update from remote peer 225c9ecec8ac4f2ca6aa89c9f5d8fc03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:06:30.030897 1900 raft_consensus.cc:1261] T 70838fcabe564bd39ba20a5dfd5aa188 P 54580f2d27444280a5f21772db02ac05 [term 1 FOLLOWER]: Refusing update from remote peer 225c9ecec8ac4f2ca6aa89c9f5d8fc03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:06:30.031987 2405 consensus_queue.cc:1035] T 70838fcabe564bd39ba20a5dfd5aa188 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c5816077f504b37892ccc03751f3120" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 12:06:30.032132 2365 tablet.cc:2369] T 70838fcabe564bd39ba20a5dfd5aa188 P 9c5816077f504b37892ccc03751f3120: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:06:30.032313 2401 consensus_queue.cc:1035] T 70838fcabe564bd39ba20a5dfd5aa188 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54580f2d27444280a5f21772db02ac05" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 45207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:06:30.043922 2411 mvcc.cc:205] Tried to move back new op lower bound from 7317299159157420032 to 7317299159060119552. Current Snapshot: MvccSnapshot[applied={T|T < 7317299159157420032}]
I20260811 12:06:30.065810 1685 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:06:30.066151 1685 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:06:30.070279 1685 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589: Sending DeleteTablet for 3 replicas of tablet 70838fcabe564bd39ba20a5dfd5aa188
I20260811 12:06:30.071486 2272 tablet_service.cc:1514] Processing DeleteTablet for tablet 70838fcabe564bd39ba20a5dfd5aa188 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:06:30 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:47060
I20260811 12:06:30.071683 2068 tablet_service.cc:1514] Processing DeleteTablet for tablet 70838fcabe564bd39ba20a5dfd5aa188 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:06:30 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:46296
I20260811 12:06:30.071965 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 1778
I20260811 12:06:30.071975 1871 tablet_service.cc:1514] Processing DeleteTablet for tablet 70838fcabe564bd39ba20a5dfd5aa188 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:06:30 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:41010
I20260811 12:06:30.074574 2424 tablet_replica.cc:330] T 70838fcabe564bd39ba20a5dfd5aa188 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03: stopping tablet replica
I20260811 12:06:30.075140 2424 raft_consensus.cc:2229] T 70838fcabe564bd39ba20a5dfd5aa188 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:06:30.076380 2424 raft_consensus.cc:2258] T 70838fcabe564bd39ba20a5dfd5aa188 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:06:30.079092 2426 tablet_replica.cc:330] T 70838fcabe564bd39ba20a5dfd5aa188 P 9c5816077f504b37892ccc03751f3120: stopping tablet replica
I20260811 12:06:30.079489 2426 raft_consensus.cc:2229] T 70838fcabe564bd39ba20a5dfd5aa188 P 9c5816077f504b37892ccc03751f3120 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:06:30.079993 2426 raft_consensus.cc:2258] T 70838fcabe564bd39ba20a5dfd5aa188 P 9c5816077f504b37892ccc03751f3120 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:06:30.080760 2425 tablet_replica.cc:330] T 70838fcabe564bd39ba20a5dfd5aa188 P 54580f2d27444280a5f21772db02ac05: stopping tablet replica
I20260811 12:06:30.081151 2425 raft_consensus.cc:2229] T 70838fcabe564bd39ba20a5dfd5aa188 P 54580f2d27444280a5f21772db02ac05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:06:30.081622 2425 raft_consensus.cc:2258] T 70838fcabe564bd39ba20a5dfd5aa188 P 54580f2d27444280a5f21772db02ac05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:06:30.083845 2425 ts_tablet_manager.cc:1905] T 70838fcabe564bd39ba20a5dfd5aa188 P 54580f2d27444280a5f21772db02ac05: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:06:30.083890 2426 ts_tablet_manager.cc:1905] T 70838fcabe564bd39ba20a5dfd5aa188 P 9c5816077f504b37892ccc03751f3120: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 12:06:30 dist-test-slave-mjjr krb5kdc[1620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.129@KRBTEST.COM
I20260811 12:06:30.089529 2424 ts_tablet_manager.cc:1905] T 70838fcabe564bd39ba20a5dfd5aa188 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:06:30.091140 2425 ts_tablet_manager.cc:1918] T 70838fcabe564bd39ba20a5dfd5aa188 P 54580f2d27444280a5f21772db02ac05: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:06:30.091374 2425 log.cc:1196] T 70838fcabe564bd39ba20a5dfd5aa188 P 54580f2d27444280a5f21772db02ac05: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-0/wal/wals/70838fcabe564bd39ba20a5dfd5aa188
I20260811 12:06:30.091990 2425 ts_tablet_manager.cc:1939] T 70838fcabe564bd39ba20a5dfd5aa188 P 54580f2d27444280a5f21772db02ac05: Deleting consensus metadata
I20260811 12:06:30.093918 1665 catalog_manager.cc:4841] TS 54580f2d27444280a5f21772db02ac05 (127.9.221.129:45207): tablet 70838fcabe564bd39ba20a5dfd5aa188 (table test-table [id=890e715fbcb34e1d8478da3b73a8d678]) successfully deleted
I20260811 12:06:30.096961 2424 ts_tablet_manager.cc:1918] T 70838fcabe564bd39ba20a5dfd5aa188 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:06:30.097115 2424 log.cc:1196] T 70838fcabe564bd39ba20a5dfd5aa188 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-1/wal/wals/70838fcabe564bd39ba20a5dfd5aa188
I20260811 12:06:30.097636 2424 ts_tablet_manager.cc:1939] T 70838fcabe564bd39ba20a5dfd5aa188 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03: Deleting consensus metadata
I20260811 12:06:30.099210 1671 catalog_manager.cc:4841] TS 225c9ecec8ac4f2ca6aa89c9f5d8fc03 (127.9.221.130:35285): tablet 70838fcabe564bd39ba20a5dfd5aa188 (table test-table [id=890e715fbcb34e1d8478da3b73a8d678]) successfully deleted
I20260811 12:06:30.101467 2390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:30.073324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41012 (local address 127.9.221.129:45207)
0811 12:06:30.073629 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:30.073638 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:30.073667 (+ 29us) server_negotiation.cc:407] Connection header received
0811 12:06:30.073737 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:30.073744 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:30.073865 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:30.073993 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:06:30.076006 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:30.077326 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:30.078594 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:30.079319 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:30.079444 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:30.085274 (+ 5830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:06:30.085287 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:06:30.085301 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:06:30.085342 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:06:30.094494 (+ 9152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:30.097851 (+ 3357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:30.097860 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:30.097864 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:30.098033 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:30.099451 (+ 1418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:30.099457 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:30.099461 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:30.099982 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:06:30.100265 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:30.100886 (+ 621us) server_negotiation.cc:299] Negotiation successful
0811 12:06:30.101107 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 12:06:30.102065 1964 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:06:30.102260 2426 ts_tablet_manager.cc:1918] T 70838fcabe564bd39ba20a5dfd5aa188 P 9c5816077f504b37892ccc03751f3120: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:06:30.102525 2426 log.cc:1196] T 70838fcabe564bd39ba20a5dfd5aa188 P 9c5816077f504b37892ccc03751f3120: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-2/wal/wals/70838fcabe564bd39ba20a5dfd5aa188
I20260811 12:06:30.103116 2426 ts_tablet_manager.cc:1939] T 70838fcabe564bd39ba20a5dfd5aa188 P 9c5816077f504b37892ccc03751f3120: Deleting consensus metadata
I20260811 12:06:30.104933 1669 catalog_manager.cc:4841] TS 9c5816077f504b37892ccc03751f3120 (127.9.221.131:41145): tablet 70838fcabe564bd39ba20a5dfd5aa188 (table test-table [id=890e715fbcb34e1d8478da3b73a8d678]) successfully deleted
I20260811 12:06:30.208199 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 1778
I20260811 12:06:30.236057 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 1980
Aug 11 12:06:30 dist-test-slave-mjjr krb5kdc[1620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.130@KRBTEST.COM
I20260811 12:06:30.254364 2389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:30.236955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46308 (local address 127.9.221.130:35285)
0811 12:06:30.237136 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:30.237144 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:30.237165 (+ 21us) server_negotiation.cc:407] Connection header received
0811 12:06:30.237396 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:30.237401 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:30.237495 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:30.237614 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:06:30.238999 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:30.240022 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:30.241583 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:30.242066 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:30.242222 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:30.247764 (+ 5542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:06:30.247784 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:06:30.247805 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:06:30.247862 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:06:30.251100 (+ 3238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:30.251980 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:30.251994 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:30.251996 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:30.252153 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:30.252561 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:30.252566 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:30.252569 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:30.253113 (+ 544us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:06:30.253453 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:30.253756 (+ 303us) server_negotiation.cc:299] Negotiation successful
0811 12:06:30.253991 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 12:06:30.255085 2157 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:06:30.369683 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 1980
I20260811 12:06:30.396155 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 2170
Aug 11 12:06:30 dist-test-slave-mjjr krb5kdc[1620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.131@KRBTEST.COM
I20260811 12:06:30.411114 2386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:30.396899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47068 (local address 127.9.221.131:41145)
0811 12:06:30.397094 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:30.397102 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:30.397124 (+ 22us) server_negotiation.cc:407] Connection header received
0811 12:06:30.397333 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:30.397339 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:30.397427 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:30.397540 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:06:30.398885 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:30.399633 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:30.400967 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:30.401387 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:30.401525 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:30.406024 (+ 4499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:06:30.406037 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:06:30.406052 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:06:30.406082 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:06:30.408731 (+ 2649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:30.409399 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:30.409406 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:30.409408 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:30.409549 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:30.409871 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:30.409874 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:30.409876 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:30.410217 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:06:30.410428 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:30.410696 (+ 268us) server_negotiation.cc:299] Negotiation successful
0811 12:06:30.410856 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 12:06:30.411831 2353 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:06:30.502828 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 2170
I20260811 12:06:30.528025 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 1646
I20260811 12:06:30.529160 1733 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:06:30.641121 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 1646
I20260811 12:06:50.672261 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:34171
--webserver_interface=127.9.221.190
--webserver_port=33093
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:43273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:34171
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:06:51.028941 2434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:06:51.029256 2434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:06:51.029417 2434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:06:51.036543 2434 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 12:06:51.036667 2434 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 12:06:51.036765 2434 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 12:06:51.036865 2434 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:06:51.036966 2434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:06:51.037074 2434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:06:51.037164 2434 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:06:51.037246 2434 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:06:51.047124 2434 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:43273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:34171
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:34171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.221.190
--webserver_port=33093
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:06:51.048383 2434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:06:51.049687 2434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:06:51.075132 2454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:06:51.075098 2457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:06:51.075187 2452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:06:51.081043 2434 server_base.cc:1034] running on GCE node
I20260811 12:06:51.081956 2434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:06:51.083535 2434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:06:51.084766 2434 hybrid_clock.cc:648] HybridClock initialized: now 1786450011084701 us; error 35 us; skew 500 ppm
I20260811 12:06:51.085171 2434 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:06:51 dist-test-slave-mjjr krb5kdc[1620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450011, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:06:51.088901 2434 init.cc:377] Logged in from keytab as kudu/127.9.221.190@KRBTEST.COM (short username kudu)
I20260811 12:06:51.090570 2434 webserver.cc:460] Webserver started at http://127.9.221.190:33093/ using document root <none> and password file <none>
I20260811 12:06:51.091344 2434 fs_manager.cc:359] Metadata directory not provided
I20260811 12:06:51.091506 2434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:06:51.097406 2434 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 12:06:51.100332 2470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:06:51.101652 2434 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:06:51.101847 2434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "c4c1846038424feca3e6885cfad8f589"
format_stamp: "Formatted at 2026-08-11 12:06:28 on dist-test-slave-mjjr"
server_key: "ecfb6a6f8a3f136d21e129edb48df75b"
server_key_iv: "55aafc83132132325cfde0295656c37a"
server_key_version: "encryptionkey@0"
I20260811 12:06:51.102929 2434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:06:51.129817 2434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:06:51.134464 2434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:06:51.134779 2434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:06:51.154316 2434 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.190:34171
I20260811 12:06:51.154359 2547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.190:34171 every 8 connection(s)
I20260811 12:06:51.155947 2434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:06:51.163103 2548 sys_catalog.cc:263] Verifying existing consensus state
I20260811 12:06:51.163856 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 2434
I20260811 12:06:51.164904 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.129:45207
--local_ip_for_outbound_sockets=127.9.221.129
--tserver_master_addrs=127.9.221.190:34171
--webserver_port=46257
--webserver_interface=127.9.221.129
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab
--principal=kudu/127.9.221.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:43273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:06:51.168516 2548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589: Bootstrap starting.
I20260811 12:06:51.190806 2548 log.cc:824] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589: Log is configured to *not* fsync() on all Append() calls
I20260811 12:06:51.203308 2548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 12:06:51.203933 2548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589: Bootstrap complete.
I20260811 12:06:51.213552 2548 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4c1846038424feca3e6885cfad8f589" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 34171 } }
I20260811 12:06:51.214402 2548 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4c1846038424feca3e6885cfad8f589, State: Initialized, Role: FOLLOWER
I20260811 12:06:51.214980 2548 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4c1846038424feca3e6885cfad8f589" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 34171 } }
I20260811 12:06:51.215229 2548 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:06:51.215361 2548 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:06:51.215577 2548 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [term 1 FOLLOWER]: Advancing to term 2
I20260811 12:06:51.219583 2548 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4c1846038424feca3e6885cfad8f589" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 34171 } }
I20260811 12:06:51.220224 2548 leader_election.cc:302] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4c1846038424feca3e6885cfad8f589; no voters:
I20260811 12:06:51.220870 2548 leader_election.cc:288] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 12:06:51.221004 2558 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 12:06:51.222384 2558 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [term 2 LEADER]: Becoming Leader. State: Replica: c4c1846038424feca3e6885cfad8f589, State: Running, Role: LEADER
I20260811 12:06:51.222977 2558 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4c1846038424feca3e6885cfad8f589" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 34171 } }
I20260811 12:06:51.223272 2548 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:06:51.225726 2560 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4c1846038424feca3e6885cfad8f589. Latest consensus state: current_term: 2 leader_uuid: "c4c1846038424feca3e6885cfad8f589" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4c1846038424feca3e6885cfad8f589" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 34171 } } }
I20260811 12:06:51.226138 2560 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [sys.catalog]: This master's current role is: LEADER
I20260811 12:06:51.226701 2559 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c4c1846038424feca3e6885cfad8f589" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4c1846038424feca3e6885cfad8f589" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 34171 } } }
I20260811 12:06:51.227030 2559 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [sys.catalog]: This master's current role is: LEADER
I20260811 12:06:51.240186 2568 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:06:51.250306 2568 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:06:51.252178 2568 catalog_manager.cc:1259] Loaded cluster ID: 7ce147b8a8934af5a9e5329269f93ede
I20260811 12:06:51.252295 2568 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:06:51.257643 2568 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:06:51.260396 2568 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589: Loaded TSK: 0
I20260811 12:06:51.309023 2568 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589: Generated new TSK 1
I20260811 12:06:51.309533 2568 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 12:06:51.546985 2555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:06:51.547292 2555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:06:51.547461 2555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:06:51.554819 2555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:06:51.554981 2555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:06:51.555151 2555 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.129
I20260811 12:06:51.565124 2555 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:43273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab
--principal=kudu/127.9.221.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.129:45207
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.221.129
--webserver_port=46257
--tserver_master_addrs=127.9.221.190:34171
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.129
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:06:51.566495 2555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:06:51.567889 2555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:06:51.582902 2602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:06:51.583081 2601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:06:51.582926 2605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:06:51.584514 2555 server_base.cc:1034] running on GCE node
I20260811 12:06:51.585158 2555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:06:51.586187 2555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:06:51.587420 2555 hybrid_clock.cc:648] HybridClock initialized: now 1786450011587355 us; error 61 us; skew 500 ppm
I20260811 12:06:51.587800 2555 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:06:51 dist-test-slave-mjjr krb5kdc[1620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450011, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:06:51.591049 2555 init.cc:377] Logged in from keytab as kudu/127.9.221.129@KRBTEST.COM (short username kudu)
I20260811 12:06:51.592680 2555 webserver.cc:460] Webserver started at http://127.9.221.129:46257/ using document root <none> and password file <none>
I20260811 12:06:51.593400 2555 fs_manager.cc:359] Metadata directory not provided
I20260811 12:06:51.593551 2555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:06:51.599174 2555 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.002s
I20260811 12:06:51.602035 2616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:06:51.603492 2555 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:06:51.603688 2555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-0/wal
uuid: "54580f2d27444280a5f21772db02ac05"
format_stamp: "Formatted at 2026-08-11 12:06:28 on dist-test-slave-mjjr"
server_key: "d3da32d3f13ebd7b30eaa096fa5738a8"
server_key_iv: "e9ff98d4c1ae70a2f0f0c454955b1a6a"
server_key_version: "encryptionkey@0"
I20260811 12:06:51.604692 2555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:06:51.640219 2555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:06:51.644595 2555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:06:51.644883 2555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:06:51.645960 2555 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:06:51.647327 2555 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:06:51.647441 2555 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:06:51.647572 2555 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:06:51.647682 2555 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:06:51.681386 2555 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.129:45207
I20260811 12:06:51.681437 2793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.129:45207 every 8 connection(s)
I20260811 12:06:51.682844 2555 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
I20260811 12:06:51.686569 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 2555
I20260811 12:06:51.687291 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.130:35285
--local_ip_for_outbound_sockets=127.9.221.130
--tserver_master_addrs=127.9.221.190:34171
--webserver_port=42069
--webserver_interface=127.9.221.130
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab
--principal=kudu/127.9.221.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:43273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 12:06:51 dist-test-slave-mjjr krb5kdc[1620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450011, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:06:51.709259 2800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:51.685662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.129:41465)
0811 12:06:51.686596 (+ 934us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:51.686637 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 12:06:51.688498 (+ 1861us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:06:51.689803 (+ 1305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:06:51.689826 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:06:51.690425 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:06:51.690937 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:51.690954 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:51.693152 (+ 2198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:51.693160 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:51.695213 (+ 2053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:51.695232 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:51.696585 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:51.696595 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:51.696903 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:51.697661 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:06:51.697700 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:06:51.700240 (+ 2540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:06:51.704829 (+ 4589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:06:51.704835 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:06:51.704844 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:06:51.705219 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:06:51.705744 (+ 525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:06:51.705751 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:06:51.705753 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:06:51.705900 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:06:51.706861 (+ 961us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:06:51.706866 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:06:51.707155 (+ 289us) client_negotiation.cc:769] Sending connection context
0811 12:06:51.707465 (+ 310us) client_negotiation.cc:240] Negotiation successful
0811 12:06:51.707982 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"thread_start_us":488,"threads_started":1}
I20260811 12:06:51.709259 2799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:51.685548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:41465 (local address 127.9.221.190:34171)
0811 12:06:51.686243 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:51.686260 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:51.687589 (+ 1329us) server_negotiation.cc:407] Connection header received
0811 12:06:51.689023 (+ 1434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:51.689035 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:51.689331 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:51.689670 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:06:51.691170 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:51.692972 (+ 1802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:51.695380 (+ 2408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:51.696331 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:51.696510 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:51.700511 (+ 4001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:06:51.700523 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:06:51.700542 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:06:51.700597 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:06:51.704686 (+ 4089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:51.705386 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:51.705397 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:51.705406 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:51.705621 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:51.706020 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:51.706027 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:51.706033 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:51.706712 (+ 679us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:06:51.706893 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:51.707719 (+ 826us) server_negotiation.cc:299] Negotiation successful
0811 12:06:51.708156 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":325,"threads_started":1}
I20260811 12:06:51.712283 2797 heartbeater.cc:347] Connected to a master server at 127.9.221.190:34171
I20260811 12:06:51.712563 2797 heartbeater.cc:464] Registering TS with master...
I20260811 12:06:51.713212 2797 heartbeater.cc:511] Master 127.9.221.190:34171 requested a full tablet report, sending...
I20260811 12:06:51.715234 2498 ts_manager.cc:194] Registered new tserver with Master: 54580f2d27444280a5f21772db02ac05 (127.9.221.129:45207)
I20260811 12:06:51.716693 2498 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.129@KRBTEST.COM'} at 127.9.221.129:41465
W20260811 12:06:52.030320 2801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:06:52.030575 2801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:06:52.030668 2801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:06:52.037707 2801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:06:52.037823 2801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:06:52.037992 2801 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.130
I20260811 12:06:52.047181 2801 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:43273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab
--principal=kudu/127.9.221.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.130:35285
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.221.130
--webserver_port=42069
--tserver_master_addrs=127.9.221.190:34171
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.130
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:06:52.048516 2801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:06:52.049778 2801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:06:52.062356 2809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:06:52.062410 2813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:06:52.062529 2810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:06:52.063848 2801 server_base.cc:1034] running on GCE node
I20260811 12:06:52.064495 2801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:06:52.065380 2801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:06:52.066649 2801 hybrid_clock.cc:648] HybridClock initialized: now 1786450012066602 us; error 49 us; skew 500 ppm
I20260811 12:06:52.067031 2801 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:06:52 dist-test-slave-mjjr krb5kdc[1620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450012, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:06:52.070176 2801 init.cc:377] Logged in from keytab as kudu/127.9.221.130@KRBTEST.COM (short username kudu)
I20260811 12:06:52.071684 2801 webserver.cc:460] Webserver started at http://127.9.221.130:42069/ using document root <none> and password file <none>
I20260811 12:06:52.072356 2801 fs_manager.cc:359] Metadata directory not provided
I20260811 12:06:52.072523 2801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:06:52.077656 2801 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260811 12:06:52.080271 2822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:06:52.081544 2801 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:06:52.081730 2801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-1/wal
uuid: "225c9ecec8ac4f2ca6aa89c9f5d8fc03"
format_stamp: "Formatted at 2026-08-11 12:06:29 on dist-test-slave-mjjr"
server_key: "6f941d3e7dcecd5716e78cc05912f543"
server_key_iv: "adf3c9a4373acd93250cde0880f14d5d"
server_key_version: "encryptionkey@0"
I20260811 12:06:52.082696 2801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:06:52.110942 2801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:06:52.115056 2801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:06:52.115345 2801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:06:52.116482 2801 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:06:52.117890 2801 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:06:52.118021 2801 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:06:52.118153 2801 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:06:52.118255 2801 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:06:52.151304 2801 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.130:35285
I20260811 12:06:52.151371 2997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.130:35285 every 8 connection(s)
I20260811 12:06:52.152781 2801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
I20260811 12:06:52.159847 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 2801
I20260811 12:06:52.160646 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.131:41145
--local_ip_for_outbound_sockets=127.9.221.131
--tserver_master_addrs=127.9.221.190:34171
--webserver_port=45199
--webserver_interface=127.9.221.131
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab
--principal=kudu/127.9.221.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:43273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 12:06:52 dist-test-slave-mjjr krb5kdc[1620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450012, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.130@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:06:52.177541 2799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:52.155610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:41733 (local address 127.9.221.190:34171)
0811 12:06:52.155854 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:52.155860 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:52.157115 (+ 1255us) server_negotiation.cc:407] Connection header received
0811 12:06:52.159722 (+ 2607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:52.159727 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:52.159829 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:52.159976 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:06:52.161909 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:52.163049 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:52.165308 (+ 2259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:52.166090 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:52.166241 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:52.170714 (+ 4473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:06:52.170723 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:06:52.170730 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:06:52.170776 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:06:52.173965 (+ 3189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:52.174836 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:52.174844 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:52.174851 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:52.175008 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:52.175440 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:52.175448 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:52.175461 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:52.175903 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:06:52.176060 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:52.176820 (+ 760us) server_negotiation.cc:299] Negotiation successful
0811 12:06:52.177092 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260811 12:06:52.178176 3000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:52.156128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.130:41733)
0811 12:06:52.156936 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:52.156981 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 12:06:52.159392 (+ 2411us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:06:52.160191 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:06:52.160202 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:06:52.160962 (+ 760us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:06:52.161710 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:52.161731 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:52.163227 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:52.163232 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:52.165136 (+ 1904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:52.165150 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:52.166323 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:52.166330 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:52.166622 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:52.167431 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:06:52.167463 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:06:52.170446 (+ 2983us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:06:52.174101 (+ 3655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:06:52.174110 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:06:52.174124 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:06:52.174672 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:06:52.175126 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:06:52.175129 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:06:52.175131 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:06:52.175313 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:06:52.176053 (+ 740us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:06:52.176061 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:06:52.176443 (+ 382us) client_negotiation.cc:769] Sending connection context
0811 12:06:52.176663 (+ 220us) client_negotiation.cc:240] Negotiation successful
0811 12:06:52.177033 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":292,"threads_started":1}
I20260811 12:06:52.179697 2998 heartbeater.cc:347] Connected to a master server at 127.9.221.190:34171
I20260811 12:06:52.180019 2998 heartbeater.cc:464] Registering TS with master...
I20260811 12:06:52.180727 2998 heartbeater.cc:511] Master 127.9.221.190:34171 requested a full tablet report, sending...
I20260811 12:06:52.182147 2498 ts_manager.cc:194] Registered new tserver with Master: 225c9ecec8ac4f2ca6aa89c9f5d8fc03 (127.9.221.130:35285)
I20260811 12:06:52.183511 2498 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.130@KRBTEST.COM'} at 127.9.221.130:41733
W20260811 12:06:52.506440 3002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:06:52.506739 3002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:06:52.506882 3002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:06:52.514210 3002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:06:52.514304 3002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:06:52.514457 3002 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.131
I20260811 12:06:52.523763 3002 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:43273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab
--principal=kudu/127.9.221.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.131:41145
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.221.131
--webserver_port=45199
--tserver_master_addrs=127.9.221.190:34171
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.131
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:06:52.524924 3002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:06:52.526180 3002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:06:52.539090 3014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:06:52.539148 3007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:06:52.539143 3009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:06:52.539987 3002 server_base.cc:1034] running on GCE node
I20260811 12:06:52.540567 3002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:06:52.541458 3002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:06:52.542686 3002 hybrid_clock.cc:648] HybridClock initialized: now 1786450012542652 us; error 47 us; skew 500 ppm
I20260811 12:06:52.542984 3002 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:06:52 dist-test-slave-mjjr krb5kdc[1620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450012, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:06:52.546188 3002 init.cc:377] Logged in from keytab as kudu/127.9.221.131@KRBTEST.COM (short username kudu)
I20260811 12:06:52.547610 3002 webserver.cc:460] Webserver started at http://127.9.221.131:45199/ using document root <none> and password file <none>
I20260811 12:06:52.548316 3002 fs_manager.cc:359] Metadata directory not provided
I20260811 12:06:52.548470 3002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:06:52.553579 3002 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 12:06:52.556159 3024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:06:52.557430 3002 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 12:06:52.557593 3002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-2/wal
uuid: "9c5816077f504b37892ccc03751f3120"
format_stamp: "Formatted at 2026-08-11 12:06:29 on dist-test-slave-mjjr"
server_key: "b3c89d8bf9581a010408e82c2c3d518b"
server_key_iv: "1c565945cf5ad91d331d3c59ca71a56e"
server_key_version: "encryptionkey@0"
I20260811 12:06:52.558557 3002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:06:52.603511 3002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:06:52.608533 3002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:06:52.608834 3002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:06:52.609928 3002 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:06:52.611300 3002 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:06:52.611431 3002 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:06:52.611563 3002 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:06:52.611673 3002 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:06:52.645890 3002 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.131:41145
I20260811 12:06:52.645944 3200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.131:41145 every 8 connection(s)
I20260811 12:06:52.647336 3002 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
I20260811 12:06:52.653689 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 3002
Aug 11 12:06:52 dist-test-slave-mjjr krb5kdc[1620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450012, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.131@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:06:52.669250 2799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:52.649850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:35909 (local address 127.9.221.190:34171)
0811 12:06:52.650073 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:52.650079 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:52.651241 (+ 1162us) server_negotiation.cc:407] Connection header received
0811 12:06:52.653014 (+ 1773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:52.653018 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:52.653084 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:52.653177 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:06:52.654984 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:52.656497 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:52.658705 (+ 2208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:52.659470 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:52.659620 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:52.663184 (+ 3564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:06:52.663192 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:06:52.663196 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:06:52.663239 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:06:52.665996 (+ 2757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:52.666726 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:52.666732 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:52.666736 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:52.666899 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:52.667319 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:52.667324 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:52.667328 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:52.667719 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:06:52.667888 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:52.668564 (+ 676us) server_negotiation.cc:299] Negotiation successful
0811 12:06:52.668828 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 12:06:52.669889 3203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:52.650377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.131:35909)
0811 12:06:52.651072 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:52.651115 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 12:06:52.652759 (+ 1644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:06:52.653392 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:06:52.653399 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:06:52.653973 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:06:52.654710 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:52.654738 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:52.656769 (+ 2031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:52.656778 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:52.658574 (+ 1796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:52.658587 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:52.659725 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:52.659731 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:52.659977 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:52.660689 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:06:52.660710 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:06:52.663000 (+ 2290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:06:52.666137 (+ 3137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:06:52.666148 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:06:52.666164 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:06:52.666608 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:06:52.667018 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:06:52.667022 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:06:52.667024 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:06:52.667202 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:06:52.667861 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:06:52.667869 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:06:52.668217 (+ 348us) client_negotiation.cc:769] Sending connection context
0811 12:06:52.668447 (+ 230us) client_negotiation.cc:240] Negotiation successful
0811 12:06:52.668799 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":264,"threads_started":1}
I20260811 12:06:52.671581 3201 heartbeater.cc:347] Connected to a master server at 127.9.221.190:34171
I20260811 12:06:52.671918 3201 heartbeater.cc:464] Registering TS with master...
I20260811 12:06:52.672255 3204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:52.654591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54020 (local address 127.9.221.190:34171)
0811 12:06:52.655058 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:52.655062 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:52.658109 (+ 3047us) server_negotiation.cc:407] Connection header received
0811 12:06:52.658317 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:52.658322 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:52.658380 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:52.658489 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:06:52.660691 (+ 2202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:52.661388 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:52.662919 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:52.663533 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:52.663674 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:52.666208 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:06:52.666213 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:06:52.666215 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:06:52.666243 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:06:52.669374 (+ 3131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:52.670048 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:52.670053 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:52.670056 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:52.670190 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:52.670661 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:52.670667 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:52.670670 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:52.671026 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:06:52.671286 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:52.671610 (+ 324us) server_negotiation.cc:299] Negotiation successful
0811 12:06:52.671874 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":256,"threads_started":1}
I20260811 12:06:52.672772 3201 heartbeater.cc:511] Master 127.9.221.190:34171 requested a full tablet report, sending...
I20260811 12:06:52.674441 2495 ts_manager.cc:194] Registered new tserver with Master: 9c5816077f504b37892ccc03751f3120 (127.9.221.131:41145)
I20260811 12:06:52.675354 2495 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.131@KRBTEST.COM'} at 127.9.221.131:35909
I20260811 12:06:52.686895 10102 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:06:52.693079 3204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:52.687881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54024 (local address 127.9.221.190:34171)
0811 12:06:52.688040 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:52.688043 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:52.688422 (+ 379us) server_negotiation.cc:407] Connection header received
0811 12:06:52.688584 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:52.688588 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:52.688703 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:52.688814 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:06:52.689212 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:52.689870 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:52.691283 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:52.691769 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:52.691889 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:52.692344 (+ 455us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:06:52.692615 (+ 271us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0811 12:06:52.692890 (+ 275us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54024: token expired
Metrics: {"server-negotiator.queue_time_us":50}
W20260811 12:06:52.693200 3204 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54024: token expired
W20260811 12:06:52.693706 2374 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.9.221.190:34171)
I20260811 12:06:52.705495 3204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:52.694194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54032 (local address 127.9.221.190:34171)
0811 12:06:52.694327 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:52.694331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:52.694434 (+ 103us) server_negotiation.cc:407] Connection header received
0811 12:06:52.694536 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:52.694539 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:52.694588 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:52.694659 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:06:52.696064 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:52.696719 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:52.697921 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:52.698273 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:52.698355 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:52.700951 (+ 2596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:06:52.700955 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:06:52.700958 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:06:52.700986 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:06:52.703284 (+ 2298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:52.703938 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:52.703942 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:52.703944 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:52.704068 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:52.704415 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:52.704418 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:52.704420 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:52.704703 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:06:52.704879 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:52.705131 (+ 252us) server_negotiation.cc:299] Negotiation successful
0811 12:06:52.705274 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 12:06:52.708820 2495 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54032:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:06:52.716002 2495 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:06:52.719727 2797 heartbeater.cc:503] Master 127.9.221.190:34171 was elected leader, sending a full tablet report...
I20260811 12:06:52.750185 3218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:52.728030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42774 (local address 127.9.221.129:45207)
0811 12:06:52.740787 (+ 12757us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:52.740799 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:52.740831 (+ 32us) server_negotiation.cc:407] Connection header received
0811 12:06:52.740944 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:52.740954 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:52.741214 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:52.741410 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:06:52.742452 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:52.744189 (+ 1737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:52.747558 (+ 3369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:52.748974 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:52.749128 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:52.749357 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:52.749662 (+ 305us) server_negotiation.cc:299] Negotiation successful
0811 12:06:52.749826 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12552,"thread_start_us":12399,"threads_started":1}
I20260811 12:06:52.750204 3221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:52.727725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:45207 (local address 127.0.0.1:42774)
0811 12:06:52.738380 (+ 10655us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:52.738427 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 12:06:52.738618 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:06:52.741717 (+ 3099us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:06:52.741728 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:06:52.741776 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:06:52.742227 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:52.742241 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:52.744373 (+ 2132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:52.744377 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:52.747185 (+ 2808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:52.747200 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:52.749195 (+ 1995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:52.749201 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:52.749428 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:52.749464 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 12:06:52.749622 (+ 158us) client_negotiation.cc:240] Negotiation successful
0811 12:06:52.749802 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10365,"thread_start_us":14671,"threads_started":1}
I20260811 12:06:52.751405 3217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:52.728045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.131:41145 (local address 127.0.0.1:44078)
0811 12:06:52.741106 (+ 13061us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:52.741130 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 12:06:52.741396 (+ 266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:06:52.744283 (+ 2887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:06:52.744287 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:06:52.744310 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:06:52.744720 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:52.744732 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:52.746402 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:52.746409 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:52.748487 (+ 2078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:52.748507 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:52.750653 (+ 2146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:52.750659 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:52.750825 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:52.750857 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 12:06:52.750946 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 12:06:52.751073 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12909,"thread_start_us":9898,"threads_started":1}
I20260811 12:06:52.752457 3223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:52.728725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44078 (local address 127.9.221.131:41145)
0811 12:06:52.743632 (+ 14907us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:52.743662 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:52.743718 (+ 56us) server_negotiation.cc:407] Connection header received
0811 12:06:52.743830 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:52.743840 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:52.744072 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:52.744515 (+ 443us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:06:52.744898 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:52.746270 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:52.748672 (+ 2402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:52.750106 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:52.750233 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:52.750471 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:52.751887 (+ 1416us) server_negotiation.cc:299] Negotiation successful
0811 12:06:52.752070 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14687,"thread_start_us":14486,"threads_started":1}
I20260811 12:06:52.752887 3222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:52.728595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.130:35285 (local address 127.0.0.1:49558)
0811 12:06:52.744147 (+ 15552us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:52.744167 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:06:52.744299 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:06:52.745260 (+ 961us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:06:52.745266 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:06:52.745296 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:06:52.745765 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:52.745776 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:52.747882 (+ 2106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:52.747890 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:52.750092 (+ 2202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:52.750105 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:52.752128 (+ 2023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:52.752135 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:52.752295 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:52.752327 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 12:06:52.752451 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 12:06:52.752567 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15416,"thread_start_us":15256,"threads_started":1}
I20260811 12:06:52.753723 3224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:52.729255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49558 (local address 127.9.221.130:35285)
0811 12:06:52.743027 (+ 13772us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:52.743039 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:52.744445 (+ 1406us) server_negotiation.cc:407] Connection header received
0811 12:06:52.744554 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:52.744561 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:52.745055 (+ 494us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:52.745246 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:06:52.745961 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:52.747697 (+ 1736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:52.750278 (+ 2581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:52.751900 (+ 1622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:52.752050 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:52.752239 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:52.752514 (+ 275us) server_negotiation.cc:299] Negotiation successful
0811 12:06:52.752678 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13598,"thread_start_us":13475,"threads_started":1}
I20260811 12:06:52.754765 2698 tablet_service.cc:1467] Processing CreateTablet for tablet 779e8851a0374106ba864b31b556dbb0 (DEFAULT_TABLE table=test-table [id=beb10c4de7ea4c8cafa3a384ce1d4b54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:06:52.756446 2698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 779e8851a0374106ba864b31b556dbb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:06:52.756109 3099 tablet_service.cc:1467] Processing CreateTablet for tablet 779e8851a0374106ba864b31b556dbb0 (DEFAULT_TABLE table=test-table [id=beb10c4de7ea4c8cafa3a384ce1d4b54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:06:52.756793 2894 tablet_service.cc:1467] Processing CreateTablet for tablet 779e8851a0374106ba864b31b556dbb0 (DEFAULT_TABLE table=test-table [id=beb10c4de7ea4c8cafa3a384ce1d4b54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:06:52.757470 3099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 779e8851a0374106ba864b31b556dbb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:06:52.757886 2894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 779e8851a0374106ba864b31b556dbb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:06:52.770538 3225 tablet_bootstrap.cc:492] T 779e8851a0374106ba864b31b556dbb0 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03: Bootstrap starting.
I20260811 12:06:52.772528 3226 tablet_bootstrap.cc:492] T 779e8851a0374106ba864b31b556dbb0 P 54580f2d27444280a5f21772db02ac05: Bootstrap starting.
I20260811 12:06:52.773381 3228 tablet_bootstrap.cc:492] T 779e8851a0374106ba864b31b556dbb0 P 9c5816077f504b37892ccc03751f3120: Bootstrap starting.
I20260811 12:06:52.775916 3225 tablet_bootstrap.cc:654] T 779e8851a0374106ba864b31b556dbb0 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03: Neither blocks nor log segments found. Creating new log.
I20260811 12:06:52.777103 3226 tablet_bootstrap.cc:654] T 779e8851a0374106ba864b31b556dbb0 P 54580f2d27444280a5f21772db02ac05: Neither blocks nor log segments found. Creating new log.
I20260811 12:06:52.777261 3225 log.cc:824] T 779e8851a0374106ba864b31b556dbb0 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03: Log is configured to *not* fsync() on all Append() calls
I20260811 12:06:52.778342 3226 log.cc:824] T 779e8851a0374106ba864b31b556dbb0 P 54580f2d27444280a5f21772db02ac05: Log is configured to *not* fsync() on all Append() calls
I20260811 12:06:52.778797 3228 tablet_bootstrap.cc:654] T 779e8851a0374106ba864b31b556dbb0 P 9c5816077f504b37892ccc03751f3120: Neither blocks nor log segments found. Creating new log.
I20260811 12:06:52.780038 3228 log.cc:824] T 779e8851a0374106ba864b31b556dbb0 P 9c5816077f504b37892ccc03751f3120: Log is configured to *not* fsync() on all Append() calls
I20260811 12:06:52.780540 3225 tablet_bootstrap.cc:492] T 779e8851a0374106ba864b31b556dbb0 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03: No bootstrap required, opened a new log
I20260811 12:06:52.780879 3225 ts_tablet_manager.cc:1397] T 779e8851a0374106ba864b31b556dbb0 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.002s
I20260811 12:06:52.781484 3226 tablet_bootstrap.cc:492] T 779e8851a0374106ba864b31b556dbb0 P 54580f2d27444280a5f21772db02ac05: No bootstrap required, opened a new log
I20260811 12:06:52.781831 3226 ts_tablet_manager.cc:1397] T 779e8851a0374106ba864b31b556dbb0 P 54580f2d27444280a5f21772db02ac05: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260811 12:06:52.782768 3228 tablet_bootstrap.cc:492] T 779e8851a0374106ba864b31b556dbb0 P 9c5816077f504b37892ccc03751f3120: No bootstrap required, opened a new log
I20260811 12:06:52.783023 3228 ts_tablet_manager.cc:1397] T 779e8851a0374106ba864b31b556dbb0 P 9c5816077f504b37892ccc03751f3120: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260811 12:06:52.790295 3228 raft_consensus.cc:348] T 779e8851a0374106ba864b31b556dbb0 P 9c5816077f504b37892ccc03751f3120 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54580f2d27444280a5f21772db02ac05" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 45207 } } peers { permanent_uuid: "9c5816077f504b37892ccc03751f3120" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41145 } } peers { permanent_uuid: "225c9ecec8ac4f2ca6aa89c9f5d8fc03" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 35285 } }
I20260811 12:06:52.790669 3228 raft_consensus.cc:374] T 779e8851a0374106ba864b31b556dbb0 P 9c5816077f504b37892ccc03751f3120 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:06:52.790782 3228 raft_consensus.cc:729] T 779e8851a0374106ba864b31b556dbb0 P 9c5816077f504b37892ccc03751f3120 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c5816077f504b37892ccc03751f3120, State: Initialized, Role: FOLLOWER
I20260811 12:06:52.791456 3228 consensus_queue.cc:260] T 779e8851a0374106ba864b31b556dbb0 P 9c5816077f504b37892ccc03751f3120 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54580f2d27444280a5f21772db02ac05" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 45207 } } peers { permanent_uuid: "9c5816077f504b37892ccc03751f3120" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41145 } } peers { permanent_uuid: "225c9ecec8ac4f2ca6aa89c9f5d8fc03" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 35285 } }
I20260811 12:06:52.792212 3225 raft_consensus.cc:348] T 779e8851a0374106ba864b31b556dbb0 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54580f2d27444280a5f21772db02ac05" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 45207 } } peers { permanent_uuid: "9c5816077f504b37892ccc03751f3120" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41145 } } peers { permanent_uuid: "225c9ecec8ac4f2ca6aa89c9f5d8fc03" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 35285 } }
I20260811 12:06:52.792721 3225 raft_consensus.cc:374] T 779e8851a0374106ba864b31b556dbb0 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:06:52.792652 3226 raft_consensus.cc:348] T 779e8851a0374106ba864b31b556dbb0 P 54580f2d27444280a5f21772db02ac05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54580f2d27444280a5f21772db02ac05" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 45207 } } peers { permanent_uuid: "9c5816077f504b37892ccc03751f3120" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41145 } } peers { permanent_uuid: "225c9ecec8ac4f2ca6aa89c9f5d8fc03" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 35285 } }
I20260811 12:06:52.793157 3226 raft_consensus.cc:374] T 779e8851a0374106ba864b31b556dbb0 P 54580f2d27444280a5f21772db02ac05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:06:52.792858 3225 raft_consensus.cc:729] T 779e8851a0374106ba864b31b556dbb0 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 225c9ecec8ac4f2ca6aa89c9f5d8fc03, State: Initialized, Role: FOLLOWER
I20260811 12:06:52.793509 3201 heartbeater.cc:503] Master 127.9.221.190:34171 was elected leader, sending a full tablet report...
I20260811 12:06:52.794234 3228 ts_tablet_manager.cc:1428] T 779e8851a0374106ba864b31b556dbb0 P 9c5816077f504b37892ccc03751f3120: Time spent starting tablet: real 0.011s user 0.007s sys 0.004s
I20260811 12:06:52.794064 3225 consensus_queue.cc:260] T 779e8851a0374106ba864b31b556dbb0 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54580f2d27444280a5f21772db02ac05" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 45207 } } peers { permanent_uuid: "9c5816077f504b37892ccc03751f3120" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41145 } } peers { permanent_uuid: "225c9ecec8ac4f2ca6aa89c9f5d8fc03" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 35285 } }
I20260811 12:06:52.794495 3226 raft_consensus.cc:729] T 779e8851a0374106ba864b31b556dbb0 P 54580f2d27444280a5f21772db02ac05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54580f2d27444280a5f21772db02ac05, State: Initialized, Role: FOLLOWER
I20260811 12:06:52.795289 2998 heartbeater.cc:503] Master 127.9.221.190:34171 was elected leader, sending a full tablet report...
I20260811 12:06:52.795169 3226 consensus_queue.cc:260] T 779e8851a0374106ba864b31b556dbb0 P 54580f2d27444280a5f21772db02ac05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54580f2d27444280a5f21772db02ac05" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 45207 } } peers { permanent_uuid: "9c5816077f504b37892ccc03751f3120" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41145 } } peers { permanent_uuid: "225c9ecec8ac4f2ca6aa89c9f5d8fc03" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 35285 } }
I20260811 12:06:52.795841 3225 ts_tablet_manager.cc:1428] T 779e8851a0374106ba864b31b556dbb0 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03: Time spent starting tablet: real 0.015s user 0.009s sys 0.005s
I20260811 12:06:52.797443 3226 ts_tablet_manager.cc:1428] T 779e8851a0374106ba864b31b556dbb0 P 54580f2d27444280a5f21772db02ac05: Time spent starting tablet: real 0.015s user 0.008s sys 0.005s
W20260811 12:06:52.898964 3202 tablet.cc:2369] T 779e8851a0374106ba864b31b556dbb0 P 9c5816077f504b37892ccc03751f3120: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:06:52.904765 2999 tablet.cc:2369] T 779e8851a0374106ba864b31b556dbb0 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:06:52.919015 3234 raft_consensus.cc:482] T 779e8851a0374106ba864b31b556dbb0 P 9c5816077f504b37892ccc03751f3120 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:06:52.919263 3234 raft_consensus.cc:504] T 779e8851a0374106ba864b31b556dbb0 P 9c5816077f504b37892ccc03751f3120 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54580f2d27444280a5f21772db02ac05" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 45207 } } peers { permanent_uuid: "9c5816077f504b37892ccc03751f3120" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41145 } } peers { permanent_uuid: "225c9ecec8ac4f2ca6aa89c9f5d8fc03" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 35285 } }
I20260811 12:06:52.920984 3234 leader_election.cc:288] T 779e8851a0374106ba864b31b556dbb0 P 9c5816077f504b37892ccc03751f3120 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 54580f2d27444280a5f21772db02ac05 (127.9.221.129:45207), 225c9ecec8ac4f2ca6aa89c9f5d8fc03 (127.9.221.130:35285)
I20260811 12:06:52.927400 3203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:52.921199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:45207 (local address 127.9.221.131:37405)
0811 12:06:52.921427 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:52.921449 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:06:52.921583 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:06:52.922063 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:06:52.922068 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:06:52.922094 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:06:52.922274 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:52.922282 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:52.923597 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:52.923604 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:52.925300 (+ 1696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:52.925317 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:52.926623 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:52.926629 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:52.926806 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:52.926851 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 12:06:52.926954 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 12:06:52.927094 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":24}
I20260811 12:06:52.927497 3218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:52.921299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:37405 (local address 127.9.221.129:45207)
0811 12:06:52.921517 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:52.921524 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:52.921547 (+ 23us) server_negotiation.cc:407] Connection header received
0811 12:06:52.921765 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:52.921789 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:52.921876 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:52.921978 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:06:52.922422 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:52.923454 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:52.925479 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:52.926444 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:52.926586 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:52.926640 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:52.927001 (+ 361us) server_negotiation.cc:299] Negotiation successful
0811 12:06:52.927239 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 12:06:52.928408 2729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "779e8851a0374106ba864b31b556dbb0" candidate_uuid: "9c5816077f504b37892ccc03751f3120" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54580f2d27444280a5f21772db02ac05" is_pre_election: true
I20260811 12:06:52.929034 2729 raft_consensus.cc:2454] T 779e8851a0374106ba864b31b556dbb0 P 54580f2d27444280a5f21772db02ac05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c5816077f504b37892ccc03751f3120 in term 0.
I20260811 12:06:52.929320 3224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:52.921729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:57749 (local address 127.9.221.130:35285)
0811 12:06:52.921973 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:52.921978 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:52.922528 (+ 550us) server_negotiation.cc:407] Connection header received
0811 12:06:52.922649 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:52.922653 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:52.922726 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:52.922825 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:06:52.924003 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:52.925061 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:52.927162 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:52.928296 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:52.928415 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:52.928463 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:52.928977 (+ 514us) server_negotiation.cc:299] Negotiation successful
0811 12:06:52.929085 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 12:06:52.929459 3244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:52.921562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.130:35285 (local address 127.9.221.131:57749)
0811 12:06:52.922422 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0811 12:06:52.922436 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 12:06:52.922540 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:06:52.922836 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:06:52.922839 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:06:52.922859 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:06:52.923831 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:52.923844 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:52.925220 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:52.925225 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:52.926982 (+ 1757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:06:52.926995 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:06:52.928496 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:06:52.928504 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:06:52.928685 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:52.928787 (+ 102us) client_negotiation.cc:769] Sending connection context
0811 12:06:52.928927 (+ 140us) client_negotiation.cc:240] Negotiation successful
0811 12:06:52.929058 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":766,"mutex_wait_us":22,"thread_start_us":515,"threads_started":1}
I20260811 12:06:52.929828 3028 leader_election.cc:302] T 779e8851a0374106ba864b31b556dbb0 P 9c5816077f504b37892ccc03751f3120 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54580f2d27444280a5f21772db02ac05, 9c5816077f504b37892ccc03751f3120; no voters:
I20260811 12:06:52.930255 3234 raft_consensus.cc:2781] T 779e8851a0374106ba864b31b556dbb0 P 9c5816077f504b37892ccc03751f3120 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:06:52.930414 3234 raft_consensus.cc:482] T 779e8851a0374106ba864b31b556dbb0 P 9c5816077f504b37892ccc03751f3120 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:06:52.930538 3234 raft_consensus.cc:3037] T 779e8851a0374106ba864b31b556dbb0 P 9c5816077f504b37892ccc03751f3120 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:06:52.930565 2926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "779e8851a0374106ba864b31b556dbb0" candidate_uuid: "9c5816077f504b37892ccc03751f3120" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "225c9ecec8ac4f2ca6aa89c9f5d8fc03" is_pre_election: true
I20260811 12:06:52.931026 2926 raft_consensus.cc:2454] T 779e8851a0374106ba864b31b556dbb0 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c5816077f504b37892ccc03751f3120 in term 0.
I20260811 12:06:52.933519 3234 raft_consensus.cc:504] T 779e8851a0374106ba864b31b556dbb0 P 9c5816077f504b37892ccc03751f3120 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54580f2d27444280a5f21772db02ac05" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 45207 } } peers { permanent_uuid: "9c5816077f504b37892ccc03751f3120" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41145 } } peers { permanent_uuid: "225c9ecec8ac4f2ca6aa89c9f5d8fc03" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 35285 } }
I20260811 12:06:52.934273 3234 leader_election.cc:288] T 779e8851a0374106ba864b31b556dbb0 P 9c5816077f504b37892ccc03751f3120 [CANDIDATE]: Term 1 election: Requested vote from peers 54580f2d27444280a5f21772db02ac05 (127.9.221.129:45207), 225c9ecec8ac4f2ca6aa89c9f5d8fc03 (127.9.221.130:35285)
I20260811 12:06:52.934646 2729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "779e8851a0374106ba864b31b556dbb0" candidate_uuid: "9c5816077f504b37892ccc03751f3120" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54580f2d27444280a5f21772db02ac05"
I20260811 12:06:52.934720 2926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "779e8851a0374106ba864b31b556dbb0" candidate_uuid: "9c5816077f504b37892ccc03751f3120" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "225c9ecec8ac4f2ca6aa89c9f5d8fc03"
I20260811 12:06:52.934891 2926 raft_consensus.cc:3037] T 779e8851a0374106ba864b31b556dbb0 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:06:52.934890 2729 raft_consensus.cc:3037] T 779e8851a0374106ba864b31b556dbb0 P 54580f2d27444280a5f21772db02ac05 [term 0 FOLLOWER]: Advancing to term 1
W20260811 12:06:52.935113 2798 tablet.cc:2369] T 779e8851a0374106ba864b31b556dbb0 P 54580f2d27444280a5f21772db02ac05: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:06:52.938135 2926 raft_consensus.cc:2454] T 779e8851a0374106ba864b31b556dbb0 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c5816077f504b37892ccc03751f3120 in term 1.
I20260811 12:06:52.938135 2729 raft_consensus.cc:2454] T 779e8851a0374106ba864b31b556dbb0 P 54580f2d27444280a5f21772db02ac05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c5816077f504b37892ccc03751f3120 in term 1.
I20260811 12:06:52.938520 3028 leader_election.cc:302] T 779e8851a0374106ba864b31b556dbb0 P 9c5816077f504b37892ccc03751f3120 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54580f2d27444280a5f21772db02ac05, 9c5816077f504b37892ccc03751f3120; no voters:
I20260811 12:06:52.938815 3234 raft_consensus.cc:2781] T 779e8851a0374106ba864b31b556dbb0 P 9c5816077f504b37892ccc03751f3120 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:06:52.939262 3234 raft_consensus.cc:686] T 779e8851a0374106ba864b31b556dbb0 P 9c5816077f504b37892ccc03751f3120 [term 1 LEADER]: Becoming Leader. State: Replica: 9c5816077f504b37892ccc03751f3120, State: Running, Role: LEADER
I20260811 12:06:52.939778 3234 consensus_queue.cc:237] T 779e8851a0374106ba864b31b556dbb0 P 9c5816077f504b37892ccc03751f3120 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54580f2d27444280a5f21772db02ac05" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 45207 } } peers { permanent_uuid: "9c5816077f504b37892ccc03751f3120" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41145 } } peers { permanent_uuid: "225c9ecec8ac4f2ca6aa89c9f5d8fc03" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 35285 } }
I20260811 12:06:52.944425 2495 catalog_manager.cc:5495] T 779e8851a0374106ba864b31b556dbb0 P 9c5816077f504b37892ccc03751f3120 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c5816077f504b37892ccc03751f3120 (127.9.221.131). New cstate: current_term: 1 leader_uuid: "9c5816077f504b37892ccc03751f3120" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54580f2d27444280a5f21772db02ac05" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 45207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c5816077f504b37892ccc03751f3120" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 41145 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "225c9ecec8ac4f2ca6aa89c9f5d8fc03" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 35285 } health_report { overall_health: UNKNOWN } } }
I20260811 12:06:52.990756 3223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:52.985347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44082 (local address 127.9.221.131:41145)
0811 12:06:52.985583 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:52.985589 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:52.985608 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:06:52.985836 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:52.985840 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:52.985920 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:52.986078 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:06:52.986458 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:52.987176 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:52.988605 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:52.989007 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:52.989145 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:52.989647 (+ 502us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:06:52.990117 (+ 470us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:06:52.990295 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:52.990417 (+ 122us) server_negotiation.cc:299] Negotiation successful
0811 12:06:52.990494 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 12:06:52.999096 2926 raft_consensus.cc:1261] T 779e8851a0374106ba864b31b556dbb0 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03 [term 1 FOLLOWER]: Refusing update from remote peer 9c5816077f504b37892ccc03751f3120: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:06:52.999099 2729 raft_consensus.cc:1261] T 779e8851a0374106ba864b31b556dbb0 P 54580f2d27444280a5f21772db02ac05 [term 1 FOLLOWER]: Refusing update from remote peer 9c5816077f504b37892ccc03751f3120: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:06:53.000253 3234 consensus_queue.cc:1035] T 779e8851a0374106ba864b31b556dbb0 P 9c5816077f504b37892ccc03751f3120 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54580f2d27444280a5f21772db02ac05" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 45207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:06:53.000587 3245 consensus_queue.cc:1035] T 779e8851a0374106ba864b31b556dbb0 P 9c5816077f504b37892ccc03751f3120 [LEADER]: Connected to new peer: Peer: permanent_uuid: "225c9ecec8ac4f2ca6aa89c9f5d8fc03" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:06:53.012204 3252 mvcc.cc:205] Tried to move back new op lower bound from 7317299253232877568 to 7317299253004931072. Current Snapshot: MvccSnapshot[applied={T|T < 7317299253232877568}]
I20260811 12:06:53.013232 3248 mvcc.cc:205] Tried to move back new op lower bound from 7317299253232877568 to 7317299253004931072. Current Snapshot: MvccSnapshot[applied={T|T < 7317299253232877568}]
I20260811 12:06:53.017961 3249 mvcc.cc:205] Tried to move back new op lower bound from 7317299253232877568 to 7317299253004931072. Current Snapshot: MvccSnapshot[applied={T|T < 7317299253232877568}]
I20260811 12:06:53.038710 2495 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54032:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:06:53.039016 2495 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54032:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:06:53.042550 2495 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589: Sending DeleteTablet for 3 replicas of tablet 779e8851a0374106ba864b31b556dbb0
I20260811 12:06:53.043710 3099 tablet_service.cc:1514] Processing DeleteTablet for tablet 779e8851a0374106ba864b31b556dbb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:06:53 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:44078
I20260811 12:06:53.043931 2894 tablet_service.cc:1514] Processing DeleteTablet for tablet 779e8851a0374106ba864b31b556dbb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:06:53 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:49558
I20260811 12:06:53.044420 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 2555
I20260811 12:06:53.044471 2698 tablet_service.cc:1514] Processing DeleteTablet for tablet 779e8851a0374106ba864b31b556dbb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:06:53 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:42774
I20260811 12:06:53.044646 3266 tablet_replica.cc:330] T 779e8851a0374106ba864b31b556dbb0 P 9c5816077f504b37892ccc03751f3120: stopping tablet replica
I20260811 12:06:53.045114 3266 raft_consensus.cc:2229] T 779e8851a0374106ba864b31b556dbb0 P 9c5816077f504b37892ccc03751f3120 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:06:53.045419 3268 tablet_replica.cc:330] T 779e8851a0374106ba864b31b556dbb0 P 54580f2d27444280a5f21772db02ac05: stopping tablet replica
I20260811 12:06:53.045835 3268 raft_consensus.cc:2229] T 779e8851a0374106ba864b31b556dbb0 P 54580f2d27444280a5f21772db02ac05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:06:53.046211 3266 raft_consensus.cc:2258] T 779e8851a0374106ba864b31b556dbb0 P 9c5816077f504b37892ccc03751f3120 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:06:53.046383 3268 raft_consensus.cc:2258] T 779e8851a0374106ba864b31b556dbb0 P 54580f2d27444280a5f21772db02ac05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:06:53.048375 3268 ts_tablet_manager.cc:1905] T 779e8851a0374106ba864b31b556dbb0 P 54580f2d27444280a5f21772db02ac05: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:06:53.048481 3266 ts_tablet_manager.cc:1905] T 779e8851a0374106ba864b31b556dbb0 P 9c5816077f504b37892ccc03751f3120: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:06:53.056102 3266 ts_tablet_manager.cc:1918] T 779e8851a0374106ba864b31b556dbb0 P 9c5816077f504b37892ccc03751f3120: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:06:53.056362 3266 log.cc:1196] T 779e8851a0374106ba864b31b556dbb0 P 9c5816077f504b37892ccc03751f3120: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-2/wal/wals/779e8851a0374106ba864b31b556dbb0
I20260811 12:06:53.056996 3266 ts_tablet_manager.cc:1939] T 779e8851a0374106ba864b31b556dbb0 P 9c5816077f504b37892ccc03751f3120: Deleting consensus metadata
I20260811 12:06:53.058851 2475 catalog_manager.cc:4841] TS 9c5816077f504b37892ccc03751f3120 (127.9.221.131:41145): tablet 779e8851a0374106ba864b31b556dbb0 (table test-table [id=beb10c4de7ea4c8cafa3a384ce1d4b54]) successfully deleted
I20260811 12:06:53.060791 3268 ts_tablet_manager.cc:1918] T 779e8851a0374106ba864b31b556dbb0 P 54580f2d27444280a5f21772db02ac05: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:06:53.061064 3268 log.cc:1196] T 779e8851a0374106ba864b31b556dbb0 P 54580f2d27444280a5f21772db02ac05: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-0/wal/wals/779e8851a0374106ba864b31b556dbb0
I20260811 12:06:53.061672 3268 ts_tablet_manager.cc:1939] T 779e8851a0374106ba864b31b556dbb0 P 54580f2d27444280a5f21772db02ac05: Deleting consensus metadata
I20260811 12:06:53.063601 2472 catalog_manager.cc:4841] TS 54580f2d27444280a5f21772db02ac05 (127.9.221.129:45207): tablet 779e8851a0374106ba864b31b556dbb0 (table test-table [id=beb10c4de7ea4c8cafa3a384ce1d4b54]) successfully deleted
I20260811 12:06:53.064822 3267 tablet_replica.cc:330] T 779e8851a0374106ba864b31b556dbb0 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03: stopping tablet replica
I20260811 12:06:53.065217 3218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:53.045557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42784 (local address 127.9.221.129:45207)
0811 12:06:53.045713 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:53.045721 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:53.045742 (+ 21us) server_negotiation.cc:407] Connection header received
0811 12:06:53.045797 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:53.045802 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:53.045898 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:53.046010 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:06:53.048108 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:53.049203 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:53.051184 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:53.051838 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:53.051976 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:53.056509 (+ 4533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:06:53.056527 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:06:53.056549 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:06:53.056601 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:06:53.061628 (+ 5027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:53.062570 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:53.062577 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:53.062581 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:53.062767 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:53.063256 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:53.063261 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:53.063263 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:53.063743 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:06:53.064039 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:53.064547 (+ 508us) server_negotiation.cc:299] Negotiation successful
0811 12:06:53.064760 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 12:06:53.065248 3267 raft_consensus.cc:2229] T 779e8851a0374106ba864b31b556dbb0 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:06:53.065753 3267 raft_consensus.cc:2258] T 779e8851a0374106ba864b31b556dbb0 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:06:53.065780 2784 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:06:53.068089 3267 ts_tablet_manager.cc:1905] T 779e8851a0374106ba864b31b556dbb0 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:06:53.074958 3267 ts_tablet_manager.cc:1918] T 779e8851a0374106ba864b31b556dbb0 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:06:53.075131 3267 log.cc:1196] T 779e8851a0374106ba864b31b556dbb0 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-1/wal/wals/779e8851a0374106ba864b31b556dbb0
I20260811 12:06:53.075811 3267 ts_tablet_manager.cc:1939] T 779e8851a0374106ba864b31b556dbb0 P 225c9ecec8ac4f2ca6aa89c9f5d8fc03: Deleting consensus metadata
I20260811 12:06:53.077657 2478 catalog_manager.cc:4841] TS 225c9ecec8ac4f2ca6aa89c9f5d8fc03 (127.9.221.130:35285): tablet 779e8851a0374106ba864b31b556dbb0 (table test-table [id=beb10c4de7ea4c8cafa3a384ce1d4b54]) successfully deleted
I20260811 12:06:53.164487 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 2555
I20260811 12:06:53.200529 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 2801
I20260811 12:06:53.217439 3224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:53.201417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49570 (local address 127.9.221.130:35285)
0811 12:06:53.201579 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:53.201583 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:53.201601 (+ 18us) server_negotiation.cc:407] Connection header received
0811 12:06:53.201792 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:53.201796 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:53.201876 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:53.201954 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:06:53.203948 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:53.204657 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:53.206727 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:53.207138 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:53.207280 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:53.210756 (+ 3476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:06:53.210768 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:06:53.210783 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:06:53.210823 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:06:53.214377 (+ 3554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:53.215196 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:53.215203 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:53.215205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:53.215324 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:53.215749 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:53.215752 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:53.215754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:53.216194 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:06:53.216496 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:53.216872 (+ 376us) server_negotiation.cc:299] Negotiation successful
0811 12:06:53.217101 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 12:06:53.218281 2989 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:06:53.307152 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 2801
I20260811 12:06:53.340391 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 3002
I20260811 12:06:53.356984 3223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:06:53.341277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44094 (local address 127.9.221.131:41145)
0811 12:06:53.341449 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 12:06:53.341453 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:06:53.341528 (+ 75us) server_negotiation.cc:407] Connection header received
0811 12:06:53.341738 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:06:53.341742 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:06:53.341828 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:06:53.341911 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:06:53.344067 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:53.344875 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:53.346774 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:06:53.347209 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:06:53.347313 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:06:53.350880 (+ 3567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:06:53.350891 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:06:53.350907 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:06:53.350945 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:06:53.354082 (+ 3137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:53.354885 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:53.354891 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:53.354894 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:53.355031 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:06:53.355476 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:06:53.355479 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:06:53.355481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:06:53.355895 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:06:53.356148 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0811 12:06:53.356475 (+ 327us) server_negotiation.cc:299] Negotiation successful
0811 12:06:53.356633 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 12:06:53.357717 3194 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:06:53.478780 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 3002
I20260811 12:06:53.508284 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 2434
I20260811 12:06:53.509473 2542 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:06:53.624426 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 2434
I20260811 12:07:13.653211 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:34171
--webserver_interface=127.9.221.190
--webserver_port=33093
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:43273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:34171
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:07:14.000614 3274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:07:14.000900 3274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:07:14.001068 3274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:07:14.008095 3274 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 12:07:14.008180 3274 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 12:07:14.008219 3274 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 12:07:14.008306 3274 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:07:14.008389 3274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:07:14.008517 3274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:07:14.008581 3274 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:07:14.008682 3274 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:07:14.018553 3274 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:43273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:34171
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:34171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.221.190
--webserver_port=33093
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:07:14.019851 3274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:07:14.021198 3274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:07:14.033041 3284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:14.033092 3280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:14.033087 3282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:07:14.034221 3274 server_base.cc:1034] running on GCE node
I20260811 12:07:14.035070 3274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:07:14.036466 3274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:07:14.037708 3274 hybrid_clock.cc:648] HybridClock initialized: now 1786450034037639 us; error 50 us; skew 500 ppm
I20260811 12:07:14.038034 3274 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:07:14 dist-test-slave-mjjr krb5kdc[1620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450034, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:07:14.044502 3274 init.cc:377] Logged in from keytab as kudu/127.9.221.190@KRBTEST.COM (short username kudu)
I20260811 12:07:14.046027 3274 webserver.cc:460] Webserver started at http://127.9.221.190:33093/ using document root <none> and password file <none>
I20260811 12:07:14.046701 3274 fs_manager.cc:359] Metadata directory not provided
I20260811 12:07:14.046790 3274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:07:14.052263 3274 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.002s
I20260811 12:07:14.055032 3295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:14.056421 3274 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:07:14.056550 3274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "c4c1846038424feca3e6885cfad8f589"
format_stamp: "Formatted at 2026-08-11 12:06:28 on dist-test-slave-mjjr"
server_key: "ecfb6a6f8a3f136d21e129edb48df75b"
server_key_iv: "55aafc83132132325cfde0295656c37a"
server_key_version: "encryptionkey@0"
I20260811 12:07:14.057466 3274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:07:14.082814 3274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:07:14.087355 3274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:07:14.087596 3274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:07:14.107227 3274 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.190:34171
I20260811 12:07:14.107265 3377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.190:34171 every 8 connection(s)
I20260811 12:07:14.108770 3274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:07:14.114058 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 3274
I20260811 12:07:14.114928 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.129:45207
--local_ip_for_outbound_sockets=127.9.221.129
--tserver_master_addrs=127.9.221.190:34171
--webserver_port=46257
--webserver_interface=127.9.221.129
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab
--principal=kudu/127.9.221.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:43273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:07:14.115785 3379 sys_catalog.cc:263] Verifying existing consensus state
I20260811 12:07:14.119160 3379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589: Bootstrap starting.
I20260811 12:07:14.139461 3379 log.cc:824] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589: Log is configured to *not* fsync() on all Append() calls
I20260811 12:07:14.157042 3379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 12:07:14.157639 3379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589: Bootstrap complete.
I20260811 12:07:14.167302 3379 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4c1846038424feca3e6885cfad8f589" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 34171 } }
I20260811 12:07:14.168334 3379 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4c1846038424feca3e6885cfad8f589, State: Initialized, Role: FOLLOWER
I20260811 12:07:14.168928 3379 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4c1846038424feca3e6885cfad8f589" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 34171 } }
I20260811 12:07:14.169180 3379 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:07:14.169325 3379 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:07:14.169509 3379 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [term 2 FOLLOWER]: Advancing to term 3
I20260811 12:07:14.173470 3379 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4c1846038424feca3e6885cfad8f589" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 34171 } }
I20260811 12:07:14.173976 3379 leader_election.cc:302] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4c1846038424feca3e6885cfad8f589; no voters:
I20260811 12:07:14.174665 3379 leader_election.cc:288] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 12:07:14.174885 3384 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 12:07:14.175957 3384 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [term 3 LEADER]: Becoming Leader. State: Replica: c4c1846038424feca3e6885cfad8f589, State: Running, Role: LEADER
I20260811 12:07:14.176601 3384 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4c1846038424feca3e6885cfad8f589" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 34171 } }
I20260811 12:07:14.176846 3379 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:07:14.179103 3385 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "c4c1846038424feca3e6885cfad8f589" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4c1846038424feca3e6885cfad8f589" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 34171 } } }
I20260811 12:07:14.179484 3385 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [sys.catalog]: This master's current role is: LEADER
I20260811 12:07:14.182662 3386 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4c1846038424feca3e6885cfad8f589. Latest consensus state: current_term: 3 leader_uuid: "c4c1846038424feca3e6885cfad8f589" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4c1846038424feca3e6885cfad8f589" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 34171 } } }
I20260811 12:07:14.183287 3386 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589 [sys.catalog]: This master's current role is: LEADER
I20260811 12:07:14.183300 3389 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:07:14.193562 3389 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:07:14.195588 3389 catalog_manager.cc:1259] Loaded cluster ID: 7ce147b8a8934af5a9e5329269f93ede
I20260811 12:07:14.195803 3389 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:07:14.201298 3389 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:07:14.203796 3389 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589: Loaded TSK: 0
I20260811 12:07:14.204061 3389 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c4c1846038424feca3e6885cfad8f589: Loaded TSK: 1
F20260811 12:07:14.205492 3389 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f1ab616f9ad google::LogMessage::Fail() at ??:0
@ 0x7f1ab617288a google::LogMessage::SendToLog() at ??:0
@ 0x7f1ab616f4de google::LogMessage::Flush() at ??:0
@ 0x7f1ab6170a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f1ab9ed37d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f1ab9ec273e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f1acbba21c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f1acbba1783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f1acbc2d2a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f1acbc2d0c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f1acbc775e9 std::function<>::operator()() at ??:0
@ 0x7f1acbba5cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f1acbba3c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f1acbc2b610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f1acbc2b439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f1ab8dbb345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f1ab8dbe91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f1ab8dbe799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f1ab8d99eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f1abc0736db start_thread at ??:0
@ 0x7f1ab500961f clone at ??:0
W20260811 12:07:14.519115 3382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:07:14.519379 3382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:07:14.519541 3382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:07:14.526926 3382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:07:14.527029 3382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:07:14.527249 3382 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.129
I20260811 12:07:14.537011 3382 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:43273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab
--principal=kudu/127.9.221.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.129:45207
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.221.129
--webserver_port=46257
--tserver_master_addrs=127.9.221.190:34171
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.129
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:07:14.538259 3382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:07:14.539541 3382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:07:14.552789 3442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:14.552757 3445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:14.552690 3443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:07:14.553632 3382 server_base.cc:1034] running on GCE node
I20260811 12:07:14.554256 3382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:07:14.555202 3382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:07:14.556424 3382 hybrid_clock.cc:648] HybridClock initialized: now 1786450034556388 us; error 50 us; skew 500 ppm
I20260811 12:07:14.556736 3382 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:07:14 dist-test-slave-mjjr krb5kdc[1620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450034, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:07:14.559815 3382 init.cc:377] Logged in from keytab as kudu/127.9.221.129@KRBTEST.COM (short username kudu)
I20260811 12:07:14.561194 3382 webserver.cc:460] Webserver started at http://127.9.221.129:46257/ using document root <none> and password file <none>
I20260811 12:07:14.561887 3382 fs_manager.cc:359] Metadata directory not provided
I20260811 12:07:14.562042 3382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:07:14.567319 3382 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:07:14.570027 3455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:14.571282 3382 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 12:07:14.571478 3382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-0/wal
uuid: "54580f2d27444280a5f21772db02ac05"
format_stamp: "Formatted at 2026-08-11 12:06:28 on dist-test-slave-mjjr"
server_key: "d3da32d3f13ebd7b30eaa096fa5738a8"
server_key_iv: "e9ff98d4c1ae70a2f0f0c454955b1a6a"
server_key_version: "encryptionkey@0"
I20260811 12:07:14.572535 3382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:07:14.585335 3382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:07:14.589082 3382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:07:14.589365 3382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:07:14.590302 3382 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:07:14.591684 3382 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:07:14.591814 3382 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:14.591949 3382 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:07:14.592041 3382 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:14.625787 3382 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.129:45207
I20260811 12:07:14.625839 3635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.129:45207 every 8 connection(s)
I20260811 12:07:14.627275 3382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
I20260811 12:07:14.632061 3641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:14.630255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.129:44941)
0811 12:07:14.630953 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:14.631196 (+ 243us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":272,"threads_started":1}
W20260811 12:07:14.632778 3638 heartbeater.cc:644] Failed to heartbeat to 127.9.221.190:34171 (0 consecutive failures): Network error: Failed to ping master at 127.9.221.190:34171: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
I20260811 12:07:14.633572 3641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:14.633261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.129:57663)
0811 12:07:14.633407 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:14.633480 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 12:07:14.634409 3641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:14.634114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.129:40325)
0811 12:07:14.634224 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:14.634294 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 12:07:14.634689 3638 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 12:07:14.635510 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 3382
I20260811 12:07:14.636162 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.130:35285
--local_ip_for_outbound_sockets=127.9.221.130
--tserver_master_addrs=127.9.221.190:34171
--webserver_port=42069
--webserver_interface=127.9.221.130
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab
--principal=kudu/127.9.221.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:43273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:07:14.979086 3642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:07:14.979341 3642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:07:14.979496 3642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:07:14.986703 3642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:07:14.986807 3642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:07:14.986995 3642 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.130
I20260811 12:07:14.996438 3642 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:43273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab
--principal=kudu/127.9.221.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.130:35285
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.221.130
--webserver_port=42069
--tserver_master_addrs=127.9.221.190:34171
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.130
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:07:14.997692 3642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:07:14.999008 3642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:07:15.011889 3647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:15.012140 3648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:15.012293 3651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:07:15.013433 3642 server_base.cc:1034] running on GCE node
I20260811 12:07:15.014076 3642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:07:15.015062 3642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:07:15.016304 3642 hybrid_clock.cc:648] HybridClock initialized: now 1786450035016246 us; error 46 us; skew 500 ppm
I20260811 12:07:15.016624 3642 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:07:15 dist-test-slave-mjjr krb5kdc[1620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450035, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:07:15.019853 3642 init.cc:377] Logged in from keytab as kudu/127.9.221.130@KRBTEST.COM (short username kudu)
I20260811 12:07:15.021215 3642 webserver.cc:460] Webserver started at http://127.9.221.130:42069/ using document root <none> and password file <none>
I20260811 12:07:15.021813 3642 fs_manager.cc:359] Metadata directory not provided
I20260811 12:07:15.021924 3642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:07:15.026989 3642 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:07:15.029608 3663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:15.030903 3642 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:07:15.031068 3642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-1/wal
uuid: "225c9ecec8ac4f2ca6aa89c9f5d8fc03"
format_stamp: "Formatted at 2026-08-11 12:06:29 on dist-test-slave-mjjr"
server_key: "6f941d3e7dcecd5716e78cc05912f543"
server_key_iv: "adf3c9a4373acd93250cde0880f14d5d"
server_key_version: "encryptionkey@0"
I20260811 12:07:15.032079 3642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:07:15.050437 3642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:07:15.054800 3642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:07:15.055089 3642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:07:15.056128 3642 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:07:15.057453 3642 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:07:15.057550 3642 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:15.057704 3642 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:07:15.057780 3642 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:15.090258 3642 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.130:35285
I20260811 12:07:15.090348 3829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.130:35285 every 8 connection(s)
I20260811 12:07:15.091781 3642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
I20260811 12:07:15.094795 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 3642
I20260811 12:07:15.095532 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.131:41145
--local_ip_for_outbound_sockets=127.9.221.131
--tserver_master_addrs=127.9.221.190:34171
--webserver_port=45199
--webserver_interface=127.9.221.131
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab
--principal=kudu/127.9.221.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:43273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:07:15.097424 3832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:15.094966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.130:33861)
0811 12:07:15.095920 (+ 954us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:15.096216 (+ 296us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":345,"threads_started":1}
W20260811 12:07:15.098198 3830 heartbeater.cc:644] Failed to heartbeat to 127.9.221.190:34171 (0 consecutive failures): Network error: Failed to ping master at 127.9.221.190:34171: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
I20260811 12:07:15.099201 3832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:15.098747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.130:41225)
0811 12:07:15.098949 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:15.099047 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 12:07:15.100281 3832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:15.099854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.130:55521)
0811 12:07:15.100016 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:15.100119 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 12:07:15.100548 3830 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 12:07:15.439930 3833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:07:15.440200 3833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:07:15.440301 3833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:07:15.447389 3833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:07:15.447494 3833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:07:15.447602 3833 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.131
I20260811 12:07:15.457858 3833 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:43273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab
--principal=kudu/127.9.221.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.131:41145
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.221.131
--webserver_port=45199
--tserver_master_addrs=127.9.221.190:34171
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.131
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:07:15.459103 3833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:07:15.460457 3833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:07:15.473639 3845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:15.473770 3842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:15.474309 3841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:07:15.474601 3833 server_base.cc:1034] running on GCE node
I20260811 12:07:15.475283 3833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:07:15.476432 3833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:07:15.477629 3833 hybrid_clock.cc:648] HybridClock initialized: now 1786450035477597 us; error 28 us; skew 500 ppm
I20260811 12:07:15.477957 3833 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:07:15 dist-test-slave-mjjr krb5kdc[1620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450035, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:07:15.481209 3833 init.cc:377] Logged in from keytab as kudu/127.9.221.131@KRBTEST.COM (short username kudu)
I20260811 12:07:15.482688 3833 webserver.cc:460] Webserver started at http://127.9.221.131:45199/ using document root <none> and password file <none>
I20260811 12:07:15.483587 3833 fs_manager.cc:359] Metadata directory not provided
I20260811 12:07:15.483783 3833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:07:15.489034 3833 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260811 12:07:15.491559 3854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:15.493156 3833 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260811 12:07:15.493331 3833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-2/wal
uuid: "9c5816077f504b37892ccc03751f3120"
format_stamp: "Formatted at 2026-08-11 12:06:29 on dist-test-slave-mjjr"
server_key: "b3c89d8bf9581a010408e82c2c3d518b"
server_key_iv: "1c565945cf5ad91d331d3c59ca71a56e"
server_key_version: "encryptionkey@0"
I20260811 12:07:15.494351 3833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:07:15.507731 3833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:07:15.511983 3833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:07:15.512266 3833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:07:15.513259 3833 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:07:15.514603 3833 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:07:15.514699 3833 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:15.514796 3833 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:07:15.514858 3833 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:15.551466 3833 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.131:41145
I20260811 12:07:15.551527 4026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.131:41145 every 8 connection(s)
I20260811 12:07:15.553081 3833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
I20260811 12:07:15.555449 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 3833
I20260811 12:07:15.558617 4031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:15.556483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.131:60461)
0811 12:07:15.557471 (+ 988us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:15.557771 (+ 300us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":663,"thread_start_us":424,"threads_started":1}
W20260811 12:07:15.559300 4027 heartbeater.cc:644] Failed to heartbeat to 127.9.221.190:34171 (0 consecutive failures): Network error: Failed to ping master at 127.9.221.190:34171: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
I20260811 12:07:15.560154 4031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:15.559853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.131:49847)
0811 12:07:15.559987 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:15.560051 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 12:07:15.561030 4031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:15.560728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.131:34961)
0811 12:07:15.560864 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:15.560931 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 12:07:15.561308 4027 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 12:07:15.636523 4032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:15.635555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.129:40409)
0811 12:07:15.636182 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:15.636296 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":310,"threads_started":1}
I20260811 12:07:16.101809 4033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:16.101028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.130:45905)
0811 12:07:16.101485 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:16.101589 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":234,"threads_started":1}
I20260811 12:07:16.562696 4034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:16.561888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.131:55837)
0811 12:07:16.562364 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:16.562479 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":228,"threads_started":1}
I20260811 12:07:16.638288 4037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:16.637521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.129:33711)
0811 12:07:16.637995 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:16.638081 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":244,"threads_started":1}
I20260811 12:07:17.103505 4038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:17.102761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.130:54813)
0811 12:07:17.103256 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:17.103339 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":250,"threads_started":1}
I20260811 12:07:17.564437 4039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:17.563554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.131:41591)
0811 12:07:17.564168 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:17.564276 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":287,"threads_started":1}
I20260811 12:07:17.639962 4040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:17.639148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.129:40285)
0811 12:07:17.639577 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:17.639717 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":228,"threads_started":1}
I20260811 12:07:18.105100 4041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:18.104409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.130:56697)
0811 12:07:18.104844 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:18.104933 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":237,"threads_started":1}
I20260811 12:07:18.565959 4042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:18.565236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.131:39097)
0811 12:07:18.565695 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:18.565776 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":266,"threads_started":1}
I20260811 12:07:18.641607 4043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:18.640913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.129:54995)
0811 12:07:18.641333 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:18.641412 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":206,"threads_started":1}
I20260811 12:07:19.106765 4044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:19.106021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.130:49393)
0811 12:07:19.106487 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:19.106572 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":236,"threads_started":1}
I20260811 12:07:19.567551 4045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:19.566805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.131:55771)
0811 12:07:19.567287 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:19.567366 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":257,"threads_started":1}
I20260811 12:07:19.643347 4046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:19.642584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.129:37989)
0811 12:07:19.643085 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:19.643167 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":262,"threads_started":1}
I20260811 12:07:20.108489 4047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:20.107712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.130:35475)
0811 12:07:20.108206 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:20.108309 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":260,"threads_started":1}
I20260811 12:07:20.569341 4048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:20.568567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.131:50719)
0811 12:07:20.569071 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:20.569166 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":262,"threads_started":1}
I20260811 12:07:20.644958 4050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:20.644215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.129:60729)
0811 12:07:20.644700 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:20.644798 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":248,"threads_started":1}
I20260811 12:07:21.110100 4053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:21.109372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.130:37645)
0811 12:07:21.109838 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:21.109933 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":228,"threads_started":1}
I20260811 12:07:21.570958 4054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:21.570217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.131:55969)
0811 12:07:21.570707 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:21.570796 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":265,"threads_started":1}
I20260811 12:07:21.646852 4055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:21.645954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.129:59591)
0811 12:07:21.646562 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:21.646642 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":316,"threads_started":1}
I20260811 12:07:22.111943 4056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:22.110979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.130:56673)
0811 12:07:22.111581 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:22.111720 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":262,"threads_started":1}
I20260811 12:07:22.572649 4057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:22.571890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.131:44685)
0811 12:07:22.572356 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:22.572438 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":197,"threads_started":1}
I20260811 12:07:22.648525 4059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:22.647849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.129:53237)
0811 12:07:22.648275 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:22.648353 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":225,"threads_started":1}
I20260811 12:07:23.113554 4061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:23.112818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.130:40015)
0811 12:07:23.113277 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:23.113386 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":249,"threads_started":1}
I20260811 12:07:23.574265 4064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:23.573547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.131:56335)
0811 12:07:23.574018 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:23.574096 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":257,"threads_started":1}
I20260811 12:07:23.650081 4065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:23.649344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.129:43841)
0811 12:07:23.649829 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:23.649911 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":252,"threads_started":1}
I20260811 12:07:24.115197 4066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:24.114443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.130:38791)
0811 12:07:24.114919 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:24.114998 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":219,"threads_started":1}
I20260811 12:07:24.576169 4067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:24.575156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.131:42355)
0811 12:07:24.575835 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:24.575958 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":287,"threads_started":1}
I20260811 12:07:24.651830 4068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:24.651056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.129:41793)
0811 12:07:24.651523 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:24.651608 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":253,"threads_started":1}
I20260811 12:07:25.117149 4069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:25.116312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.130:60433)
0811 12:07:25.116860 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:25.116989 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":275,"threads_started":1}
I20260811 12:07:25.577924 4070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:25.577151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.131:46591)
0811 12:07:25.577674 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:25.577759 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":320,"threads_started":1}
I20260811 12:07:25.653604 4071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:25.652891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.129:41717)
0811 12:07:25.653318 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:25.653420 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":209,"threads_started":1}
I20260811 12:07:26.118933 4072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:26.118132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.130:45093)
0811 12:07:26.118627 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:26.118717 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":274,"threads_started":1}
I20260811 12:07:26.579598 4073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:26.578836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.131:42449)
0811 12:07:26.579313 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:26.579403 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":256,"threads_started":1}
I20260811 12:07:26.655339 4076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:26.654632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.129:47213)
0811 12:07:26.655099 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:26.655178 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":262,"threads_started":1}
I20260811 12:07:27.120564 4077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:27.119862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.130:60409)
0811 12:07:27.120327 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:27.120407 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":225,"threads_started":1}
I20260811 12:07:27.581382 4078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:27.580602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.131:33179)
0811 12:07:27.581127 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:27.581217 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":298,"threads_started":1}
I20260811 12:07:27.657097 4079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:27.656238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.129:55457)
0811 12:07:27.656761 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:27.656864 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":254,"threads_started":1}
I20260811 12:07:28.122205 4080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:28.121364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.130:56735)
0811 12:07:28.121922 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:28.122015 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":262,"threads_started":1}
I20260811 12:07:28.583026 4081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:28.582247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.131:59989)
0811 12:07:28.582718 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:28.582836 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":245,"threads_started":1}
I20260811 12:07:28.658825 4082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:28.658047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.129:50271)
0811 12:07:28.658545 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:28.658630 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":256,"threads_started":1}
I20260811 12:07:29.123944 4083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:29.123089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.130:48231)
0811 12:07:29.123610 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:29.123754 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":329,"threads_started":1}
I20260811 12:07:29.584720 4084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:29.584006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.131:50653)
0811 12:07:29.584446 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:29.584547 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":234,"threads_started":1}
I20260811 12:07:29.660735 4085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:29.660042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.129:58737)
0811 12:07:29.660472 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:29.660558 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":221,"threads_started":1}
I20260811 12:07:30.126055 4086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:30.125212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.130:35441)
0811 12:07:30.125784 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:30.125871 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":315,"threads_started":1}
I20260811 12:07:30.563482 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 3382
I20260811 12:07:30.581905 4087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:30.564814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45314 (local address 127.9.221.129:45207)
0811 12:07:30.565497 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:30.565522 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:30.565576 (+ 54us) server_negotiation.cc:407] Connection header received
0811 12:07:30.567278 (+ 1702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:30.567287 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:30.567790 (+ 503us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:30.568086 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:30.569515 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:30.570727 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:30.572065 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:30.572692 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:30.572827 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:30.575757 (+ 2930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:30.575766 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:30.575780 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:07:30.575820 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:30.578670 (+ 2850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:30.579286 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:30.579292 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:30.579298 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:30.579471 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:30.579950 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:30.579953 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:30.579955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:30.580612 (+ 657us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:30.580851 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:30.581252 (+ 401us) server_negotiation.cc:299] Negotiation successful
0811 12:07:30.581559 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":304,"threads_started":1}
I20260811 12:07:30.583298 3625 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:07:30.586982 4091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:30.586061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:34171 (local address 127.9.221.131:42009)
0811 12:07:30.586607 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:30.586720 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.190:34171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":271,"threads_started":1}
I20260811 12:07:30.669561 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 3382
I20260811 12:07:30.697808 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 3642
I20260811 12:07:30.715830 4094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:30.698890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46482 (local address 127.9.221.130:35285)
0811 12:07:30.699533 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:30.699556 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:30.699595 (+ 39us) server_negotiation.cc:407] Connection header received
0811 12:07:30.701241 (+ 1646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:30.701252 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:30.701652 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:30.701951 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:30.703305 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:30.704696 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:30.705960 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:30.706614 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:30.706757 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:30.709433 (+ 2676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:30.709440 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:30.709453 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:07:30.709487 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:30.712607 (+ 3120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:30.713250 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:30.713257 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:30.713263 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:30.713458 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:30.713867 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:30.713871 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:30.713873 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:30.714421 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:30.714643 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:30.715005 (+ 362us) server_negotiation.cc:299] Negotiation successful
0811 12:07:30.715446 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":310,"threads_started":1}
I20260811 12:07:30.717156 3823 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:07:30.801671 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 3642
I20260811 12:07:30.828140 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 3833
I20260811 12:07:30.845963 4099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:30.829135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32898 (local address 127.9.221.131:41145)
0811 12:07:30.829770 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:30.829788 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:30.829830 (+ 42us) server_negotiation.cc:407] Connection header received
0811 12:07:30.831483 (+ 1653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:30.831492 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:30.831994 (+ 502us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:30.832366 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:30.833876 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:30.835087 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:30.836523 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:30.837227 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:30.837449 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:30.840115 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:30.840122 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:30.840136 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:07:30.840169 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:30.842818 (+ 2649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:30.843466 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:30.843472 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:30.843478 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:30.843632 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:30.844054 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:30.844061 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:30.844064 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:30.844540 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:30.844847 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:30.845291 (+ 444us) server_negotiation.cc:299] Negotiation successful
0811 12:07:30.845659 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":309,"threads_started":1}
I20260811 12:07:30.847348 4021 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:07:30.930100 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 3833
2026-08-11T12:07:30Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64885 ms)
[----------] 30 tests from SecurityITest (307744 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:07:31 dist-test-slave-mjjr krb5kdc[4103](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:07:31 dist-test-slave-mjjr krb5kdc[4103](info): set up 2 sockets
Aug 11 12:07:31 dist-test-slave-mjjr krb5kdc[4103](info): commencing operation
krb5kdc: starting...
W20260811 12:07:32.278766 10102 mini_kdc.cc:121] Time spent starting KDC: real 1.301s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Aug 11 12:07:32 dist-test-slave-mjjr krb5kdc[4103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:07:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:07:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.221.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
I20260811 12:07:32.558869 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:35693
--webserver_interface=127.9.221.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:40995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:35693
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:07:32.898429 4122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:07:32.898803 4122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:07:32.898975 4122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:07:32.906414 4122 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:07:32.906620 4122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:07:32.906733 4122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:07:32.906841 4122 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:07:32.906936 4122 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:07:32.916492 4122 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:40995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:35693
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:35693
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.221.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:07:32.917788 4122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:07:32.919119 4122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:07:32.931536 4133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:32.931491 4130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:32.931689 4128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:07:32.932310 4122 server_base.cc:1034] running on GCE node
I20260811 12:07:32.933106 4122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:07:32.934474 4122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:07:32.935738 4122 hybrid_clock.cc:648] HybridClock initialized: now 1786450052935691 us; error 53 us; skew 500 ppm
I20260811 12:07:32.936081 4122 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:07:32 dist-test-slave-mjjr krb5kdc[4103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450052, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:07:32.939396 4122 init.cc:377] Logged in from keytab as kudu/127.9.221.190@KRBTEST.COM (short username kudu)
I20260811 12:07:32.941092 4122 webserver.cc:460] Webserver started at http://127.9.221.190:40871/ using document root <none> and password file <none>
I20260811 12:07:32.941793 4122 fs_manager.cc:359] Metadata directory not provided
I20260811 12:07:32.941949 4122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:07:32.942258 4122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:07:32.945345 4122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "34e1530cf9064b18967c96f3fc145ae9"
format_stamp: "Formatted at 2026-08-11 12:07:32 on dist-test-slave-mjjr"
server_key: "1817bb7762ed4b934c6d35571837746b"
server_key_iv: "d06ead89108c6347eb53a65e188b690c"
server_key_version: "encryptionkey@0"
I20260811 12:07:32.946100 4122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "34e1530cf9064b18967c96f3fc145ae9"
format_stamp: "Formatted at 2026-08-11 12:07:32 on dist-test-slave-mjjr"
server_key: "1817bb7762ed4b934c6d35571837746b"
server_key_iv: "d06ead89108c6347eb53a65e188b690c"
server_key_version: "encryptionkey@0"
I20260811 12:07:32.951365 4122 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 12:07:32.954749 4142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:32.956103 4122 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:07:32.956276 4122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "34e1530cf9064b18967c96f3fc145ae9"
format_stamp: "Formatted at 2026-08-11 12:07:32 on dist-test-slave-mjjr"
server_key: "1817bb7762ed4b934c6d35571837746b"
server_key_iv: "d06ead89108c6347eb53a65e188b690c"
server_key_version: "encryptionkey@0"
I20260811 12:07:32.956470 4122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:07:32.978346 4122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:07:32.982349 4122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:07:32.982640 4122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:07:33.002210 4122 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.190:35693
I20260811 12:07:33.002301 4217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.190:35693 every 8 connection(s)
I20260811 12:07:33.003621 4122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:07:33.007511 4219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:07:33.008620 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 4122
I20260811 12:07:33.008849 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:07:33.009523 10102 external_mini_cluster.cc:1428] Setting key 323d915d48c761b966471f7d321d5e41
I20260811 12:07:33.022756 4219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34e1530cf9064b18967c96f3fc145ae9: Bootstrap starting.
Aug 11 12:07:33 dist-test-slave-mjjr krb5kdc[4103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:07:33.026201 4219 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34e1530cf9064b18967c96f3fc145ae9: Neither blocks nor log segments found. Creating new log.
I20260811 12:07:33.027583 4219 log.cc:824] T 00000000000000000000000000000000 P 34e1530cf9064b18967c96f3fc145ae9: Log is configured to *not* fsync() on all Append() calls
I20260811 12:07:33.031243 4219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34e1530cf9064b18967c96f3fc145ae9: No bootstrap required, opened a new log
I20260811 12:07:33.036130 4224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:33.011615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36886 (local address 127.9.221.190:35693)
0811 12:07:33.012563 (+ 948us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:33.012580 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:33.012635 (+ 55us) server_negotiation.cc:407] Connection header received
0811 12:07:33.014725 (+ 2090us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:33.014754 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:33.015262 (+ 508us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:33.015786 (+ 524us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:33.017198 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:33.019311 (+ 2113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:33.021350 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:33.022473 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:33.022676 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:33.026934 (+ 4258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:33.026945 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:33.026963 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:07:33.027016 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:33.031265 (+ 4249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:33.032085 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:33.032096 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:33.032110 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:33.032331 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:33.032709 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:33.032723 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:33.032727 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:33.033463 (+ 736us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:33.033831 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:33.034168 (+ 337us) server_negotiation.cc:299] Negotiation successful
0811 12:07:33.034644 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":656,"thread_start_us":380,"threads_started":1}
I20260811 12:07:33.041693 4219 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34e1530cf9064b18967c96f3fc145ae9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34e1530cf9064b18967c96f3fc145ae9" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 35693 } }
I20260811 12:07:33.042058 4219 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34e1530cf9064b18967c96f3fc145ae9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:07:33.042145 4219 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34e1530cf9064b18967c96f3fc145ae9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34e1530cf9064b18967c96f3fc145ae9, State: Initialized, Role: FOLLOWER
I20260811 12:07:33.042713 4219 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34e1530cf9064b18967c96f3fc145ae9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34e1530cf9064b18967c96f3fc145ae9" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 35693 } }
I20260811 12:07:33.042941 4219 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34e1530cf9064b18967c96f3fc145ae9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:07:33.043056 4219 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34e1530cf9064b18967c96f3fc145ae9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:07:33.043181 4219 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34e1530cf9064b18967c96f3fc145ae9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:07:33.045709 4219 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34e1530cf9064b18967c96f3fc145ae9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34e1530cf9064b18967c96f3fc145ae9" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 35693 } }
I20260811 12:07:33.046198 4219 leader_election.cc:302] T 00000000000000000000000000000000 P 34e1530cf9064b18967c96f3fc145ae9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34e1530cf9064b18967c96f3fc145ae9; no voters:
I20260811 12:07:33.046787 4219 leader_election.cc:288] T 00000000000000000000000000000000 P 34e1530cf9064b18967c96f3fc145ae9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:07:33.046937 4228 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34e1530cf9064b18967c96f3fc145ae9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:07:33.047624 4228 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34e1530cf9064b18967c96f3fc145ae9 [term 1 LEADER]: Becoming Leader. State: Replica: 34e1530cf9064b18967c96f3fc145ae9, State: Running, Role: LEADER
I20260811 12:07:33.048365 4228 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34e1530cf9064b18967c96f3fc145ae9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34e1530cf9064b18967c96f3fc145ae9" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 35693 } }
I20260811 12:07:33.049021 4219 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34e1530cf9064b18967c96f3fc145ae9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:07:33.051313 4230 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34e1530cf9064b18967c96f3fc145ae9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "34e1530cf9064b18967c96f3fc145ae9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34e1530cf9064b18967c96f3fc145ae9" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 35693 } } }
I20260811 12:07:33.051705 4230 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34e1530cf9064b18967c96f3fc145ae9 [sys.catalog]: This master's current role is: LEADER
I20260811 12:07:33.052296 4229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34e1530cf9064b18967c96f3fc145ae9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34e1530cf9064b18967c96f3fc145ae9. Latest consensus state: current_term: 1 leader_uuid: "34e1530cf9064b18967c96f3fc145ae9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34e1530cf9064b18967c96f3fc145ae9" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 35693 } } }
I20260811 12:07:33.053452 4229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34e1530cf9064b18967c96f3fc145ae9 [sys.catalog]: This master's current role is: LEADER
I20260811 12:07:33.054085 4236 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:07:33.059020 4236 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:07:33.067996 4236 catalog_manager.cc:1347] Generated new cluster ID: c3e449d4f20d43359606463c09aae005
I20260811 12:07:33.068111 4236 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:07:33.084039 4236 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:07:33.085367 4236 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:07:33.097103 4236 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34e1530cf9064b18967c96f3fc145ae9: Generated new TSK 0
I20260811 12:07:33.097911 4236 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.221.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab.
Entry for principal kudu/127.9.221.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab.
I20260811 12:07:33.177802 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.129:0
--local_ip_for_outbound_sockets=127.9.221.129
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:35693
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab
--principal=kudu/127.9.221.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:40995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:07:33.517992 4259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:07:33.518239 4259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:07:33.518396 4259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:07:33.525454 4259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:07:33.525568 4259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:07:33.525758 4259 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.129
I20260811 12:07:33.535362 4259 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:40995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab
--principal=kudu/127.9.221.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:35693
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.129
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:07:33.536566 4259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:07:33.537781 4259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:07:33.551127 4266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:33.551482 4269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:33.551101 4265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:07:33.552170 4259 server_base.cc:1034] running on GCE node
I20260811 12:07:33.552722 4259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:07:33.553665 4259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:07:33.554901 4259 hybrid_clock.cc:648] HybridClock initialized: now 1786450053554818 us; error 77 us; skew 500 ppm
I20260811 12:07:33.555238 4259 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:07:33 dist-test-slave-mjjr krb5kdc[4103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450053, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:07:33.558447 4259 init.cc:377] Logged in from keytab as kudu/127.9.221.129@KRBTEST.COM (short username kudu)
I20260811 12:07:33.560022 4259 webserver.cc:460] Webserver started at http://127.9.221.129:44393/ using document root <none> and password file <none>
I20260811 12:07:33.560711 4259 fs_manager.cc:359] Metadata directory not provided
I20260811 12:07:33.560868 4259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:07:33.561196 4259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:07:33.564039 4259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-0/data/instance:
uuid: "3e7bbd43a28845b0acd429692a7e27bb"
format_stamp: "Formatted at 2026-08-11 12:07:33 on dist-test-slave-mjjr"
server_key: "9ff64b2cd8b1626260846836c24f476a"
server_key_iv: "dab621f69930d8516ff384d901301438"
server_key_version: "encryptionkey@0"
I20260811 12:07:33.564776 4259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance:
uuid: "3e7bbd43a28845b0acd429692a7e27bb"
format_stamp: "Formatted at 2026-08-11 12:07:33 on dist-test-slave-mjjr"
server_key: "9ff64b2cd8b1626260846836c24f476a"
server_key_iv: "dab621f69930d8516ff384d901301438"
server_key_version: "encryptionkey@0"
I20260811 12:07:33.569792 4259 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:07:33.573210 4280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:33.574456 4259 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260811 12:07:33.574636 4259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-0/wal
uuid: "3e7bbd43a28845b0acd429692a7e27bb"
format_stamp: "Formatted at 2026-08-11 12:07:33 on dist-test-slave-mjjr"
server_key: "9ff64b2cd8b1626260846836c24f476a"
server_key_iv: "dab621f69930d8516ff384d901301438"
server_key_version: "encryptionkey@0"
I20260811 12:07:33.574844 4259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:07:33.587808 4259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:07:33.591900 4259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:07:33.592185 4259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:07:33.593056 4259 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:07:33.594305 4259 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:07:33.594414 4259 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:33.594559 4259 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:07:33.594637 4259 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:33.631525 4259 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.129:38297
I20260811 12:07:33.631556 4472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.129:38297 every 8 connection(s)
I20260811 12:07:33.633009 4259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
I20260811 12:07:33.636706 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 4259
I20260811 12:07:33.636942 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance
I20260811 12:07:33.637749 10102 external_mini_cluster.cc:1428] Setting key b5dc6106f29b48484aae421ce8656d40
Aug 11 12:07:33 dist-test-slave-mjjr krb5kdc[4103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450053, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:07:33.655082 4478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:33.635658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:59095 (local address 127.9.221.190:35693)
0811 12:07:33.636112 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:33.636117 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:33.637262 (+ 1145us) server_negotiation.cc:407] Connection header received
0811 12:07:33.639203 (+ 1941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:33.639207 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:33.639268 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:33.639350 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:33.641334 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:33.642057 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:33.644527 (+ 2470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:33.644937 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:33.645038 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:33.649342 (+ 4304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:33.649347 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:33.649350 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:07:33.649381 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:33.652064 (+ 2683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:33.652769 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:33.652779 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:33.652783 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:33.652900 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:33.653346 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:33.653352 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:33.653358 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:33.653651 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:33.653803 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:33.654494 (+ 691us) server_negotiation.cc:299] Negotiation successful
0811 12:07:33.654719 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":221,"threads_started":1}
I20260811 12:07:33.656088 4479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:33.636118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:35693 (local address 127.9.221.129:59095)
0811 12:07:33.637081 (+ 963us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:33.637157 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0811 12:07:33.638919 (+ 1762us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:07:33.639601 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:07:33.639609 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:07:33.640344 (+ 735us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:07:33.641134 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:33.641162 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:33.642221 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:33.642228 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:33.644375 (+ 2147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:33.644403 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:33.645167 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:33.645177 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:33.645520 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:33.646245 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:07:33.646278 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:07:33.649093 (+ 2815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:07:33.652212 (+ 3119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:07:33.652220 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:07:33.652234 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:07:33.652645 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:07:33.653006 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:07:33.653011 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:07:33.653015 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:07:33.653217 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:07:33.653778 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:07:33.653785 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:07:33.654104 (+ 319us) client_negotiation.cc:769] Sending connection context
0811 12:07:33.654377 (+ 273us) client_negotiation.cc:240] Negotiation successful
0811 12:07:33.654925 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":634,"thread_start_us":427,"threads_started":1}
I20260811 12:07:33.657682 4475 heartbeater.cc:347] Connected to a master server at 127.9.221.190:35693
I20260811 12:07:33.658006 4475 heartbeater.cc:464] Registering TS with master...
I20260811 12:07:33.658942 4475 heartbeater.cc:511] Master 127.9.221.190:35693 requested a full tablet report, sending...
I20260811 12:07:33.661375 4171 ts_manager.cc:194] Registered new tserver with Master: 3e7bbd43a28845b0acd429692a7e27bb (127.9.221.129:38297)
I20260811 12:07:33.663633 4171 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.129@KRBTEST.COM'} at 127.9.221.129:59095
WARNING: no policy specified for kudu/127.9.221.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab.
Entry for principal kudu/127.9.221.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab.
I20260811 12:07:33.704941 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.130:0
--local_ip_for_outbound_sockets=127.9.221.130
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:35693
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab
--principal=kudu/127.9.221.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:40995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:07:34.044519 4483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:07:34.044783 4483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:07:34.044878 4483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:07:34.051982 4483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:07:34.052085 4483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:07:34.052202 4483 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.130
I20260811 12:07:34.062026 4483 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:40995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab
--principal=kudu/127.9.221.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:35693
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.130
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:07:34.063198 4483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:07:34.064528 4483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:07:34.077569 4490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:34.077543 4492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:34.077625 4495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:07:34.078361 4483 server_base.cc:1034] running on GCE node
I20260811 12:07:34.078958 4483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:07:34.080055 4483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:07:34.081291 4483 hybrid_clock.cc:648] HybridClock initialized: now 1786450054081222 us; error 52 us; skew 500 ppm
I20260811 12:07:34.081604 4483 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:07:34 dist-test-slave-mjjr krb5kdc[4103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450054, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:07:34.089022 4483 init.cc:377] Logged in from keytab as kudu/127.9.221.130@KRBTEST.COM (short username kudu)
I20260811 12:07:34.090371 4483 webserver.cc:460] Webserver started at http://127.9.221.130:34167/ using document root <none> and password file <none>
I20260811 12:07:34.091032 4483 fs_manager.cc:359] Metadata directory not provided
I20260811 12:07:34.091171 4483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:07:34.091481 4483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:07:34.094311 4483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-1/data/instance:
uuid: "a9c5164dba7449cabf766a1d5ed1f2df"
format_stamp: "Formatted at 2026-08-11 12:07:34 on dist-test-slave-mjjr"
server_key: "145557e8dc3992cda47e3f921c1ab540"
server_key_iv: "2e1bf064dda55962b9d538d6534dbda8"
server_key_version: "encryptionkey@0"
I20260811 12:07:34.095018 4483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance:
uuid: "a9c5164dba7449cabf766a1d5ed1f2df"
format_stamp: "Formatted at 2026-08-11 12:07:34 on dist-test-slave-mjjr"
server_key: "145557e8dc3992cda47e3f921c1ab540"
server_key_iv: "2e1bf064dda55962b9d538d6534dbda8"
server_key_version: "encryptionkey@0"
I20260811 12:07:34.100215 4483 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260811 12:07:34.103487 4504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:34.104775 4483 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:07:34.104938 4483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-1/wal
uuid: "a9c5164dba7449cabf766a1d5ed1f2df"
format_stamp: "Formatted at 2026-08-11 12:07:34 on dist-test-slave-mjjr"
server_key: "145557e8dc3992cda47e3f921c1ab540"
server_key_iv: "2e1bf064dda55962b9d538d6534dbda8"
server_key_version: "encryptionkey@0"
I20260811 12:07:34.105124 4483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:07:34.137849 4483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:07:34.142133 4483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:07:34.142419 4483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:07:34.143429 4483 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:07:34.144841 4483 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:07:34.144917 4483 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:34.145089 4483 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:07:34.145150 4483 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:34.180121 4483 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.130:36191
I20260811 12:07:34.180197 4675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.130:36191 every 8 connection(s)
I20260811 12:07:34.181437 4483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
I20260811 12:07:34.182945 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 4483
I20260811 12:07:34.183166 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance
I20260811 12:07:34.183633 10102 external_mini_cluster.cc:1428] Setting key 3e7f7dc2f613b8e78e5415b836309f6a
Aug 11 12:07:34 dist-test-slave-mjjr krb5kdc[4103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450054, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.130@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:07:34.201766 4678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:34.184132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:41581 (local address 127.9.221.190:35693)
0811 12:07:34.184666 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:34.184671 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:34.186040 (+ 1369us) server_negotiation.cc:407] Connection header received
0811 12:07:34.187665 (+ 1625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:34.187671 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:34.187758 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:34.187877 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:34.189315 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:34.190037 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:34.191570 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:34.192069 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:34.192205 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:34.195788 (+ 3583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:34.195793 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:34.195796 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:07:34.195826 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:34.198411 (+ 2585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:34.199096 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:34.199104 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:34.199110 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:34.199247 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:34.199702 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:34.199709 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:34.199714 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:34.200105 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:34.200257 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:34.201121 (+ 864us) server_negotiation.cc:299] Negotiation successful
0811 12:07:34.201361 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":262,"threads_started":1}
I20260811 12:07:34.202773 4679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:34.184823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:35693 (local address 127.9.221.130:41581)
0811 12:07:34.185624 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:34.185676 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 12:07:34.187372 (+ 1696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:07:34.188113 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:07:34.188120 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:07:34.188608 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:07:34.189145 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:34.189161 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:34.190197 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:34.190201 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:34.191424 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:34.191436 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:34.192308 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:34.192316 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:34.192547 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:34.193230 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:07:34.193255 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:07:34.195529 (+ 2274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:07:34.198555 (+ 3026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:07:34.198564 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:07:34.198578 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:07:34.198974 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:07:34.199368 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:07:34.199372 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:07:34.199376 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:07:34.199569 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:07:34.200342 (+ 773us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:07:34.200350 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:07:34.200730 (+ 380us) client_negotiation.cc:769] Sending connection context
0811 12:07:34.201047 (+ 317us) client_negotiation.cc:240] Negotiation successful
0811 12:07:34.201538 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":349,"threads_started":1}
I20260811 12:07:34.204274 4676 heartbeater.cc:347] Connected to a master server at 127.9.221.190:35693
I20260811 12:07:34.204597 4676 heartbeater.cc:464] Registering TS with master...
I20260811 12:07:34.205511 4676 heartbeater.cc:511] Master 127.9.221.190:35693 requested a full tablet report, sending...
I20260811 12:07:34.207127 4171 ts_manager.cc:194] Registered new tserver with Master: a9c5164dba7449cabf766a1d5ed1f2df (127.9.221.130:36191)
I20260811 12:07:34.208431 4171 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.130@KRBTEST.COM'} at 127.9.221.130:41581
WARNING: no policy specified for kudu/127.9.221.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab.
Entry for principal kudu/127.9.221.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab.
I20260811 12:07:34.250921 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.131:0
--local_ip_for_outbound_sockets=127.9.221.131
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:35693
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab
--principal=kudu/127.9.221.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:40995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:07:34.589583 4683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:07:34.589896 4683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:07:34.590070 4683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:07:34.597268 4683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:07:34.597414 4683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:07:34.597595 4683 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.131
I20260811 12:07:34.607232 4683 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:40995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab
--principal=kudu/127.9.221.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:35693
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.131
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:07:34.608640 4683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:07:34.610119 4683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:07:34.623262 4690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:34.623193 4694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:34.623236 4691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:07:34.624035 4683 server_base.cc:1034] running on GCE node
I20260811 12:07:34.624635 4683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:07:34.625490 4683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:07:34.626731 4683 hybrid_clock.cc:648] HybridClock initialized: now 1786450054626697 us; error 46 us; skew 500 ppm
I20260811 12:07:34.627125 4683 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:07:34 dist-test-slave-mjjr krb5kdc[4103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450054, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:07:34.630378 4683 init.cc:377] Logged in from keytab as kudu/127.9.221.131@KRBTEST.COM (short username kudu)
I20260811 12:07:34.631983 4683 webserver.cc:460] Webserver started at http://127.9.221.131:43043/ using document root <none> and password file <none>
I20260811 12:07:34.632660 4683 fs_manager.cc:359] Metadata directory not provided
I20260811 12:07:34.632817 4683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:07:34.633134 4683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:07:34.636011 4683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-2/data/instance:
uuid: "a64bb7876be44a6e8c695478cbd115d8"
format_stamp: "Formatted at 2026-08-11 12:07:34 on dist-test-slave-mjjr"
server_key: "ba0a5418127b9da88ac159b5eb557881"
server_key_iv: "2453f34daea7a5517d32c1eb77459699"
server_key_version: "encryptionkey@0"
I20260811 12:07:34.636772 4683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance:
uuid: "a64bb7876be44a6e8c695478cbd115d8"
format_stamp: "Formatted at 2026-08-11 12:07:34 on dist-test-slave-mjjr"
server_key: "ba0a5418127b9da88ac159b5eb557881"
server_key_iv: "2453f34daea7a5517d32c1eb77459699"
server_key_version: "encryptionkey@0"
I20260811 12:07:34.641687 4683 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 12:07:34.645049 4706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:34.646278 4683 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260811 12:07:34.646454 4683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-2/wal
uuid: "a64bb7876be44a6e8c695478cbd115d8"
format_stamp: "Formatted at 2026-08-11 12:07:34 on dist-test-slave-mjjr"
server_key: "ba0a5418127b9da88ac159b5eb557881"
server_key_iv: "2453f34daea7a5517d32c1eb77459699"
server_key_version: "encryptionkey@0"
I20260811 12:07:34.646661 4683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:07:34.663714 4683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:07:34.667357 4475 heartbeater.cc:503] Master 127.9.221.190:35693 was elected leader, sending a full tablet report...
I20260811 12:07:34.668210 4683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:07:34.668497 4683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:07:34.669549 4683 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:07:34.670881 4683 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:07:34.670984 4683 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:34.671137 4683 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:07:34.671211 4683 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:34.705441 4683 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.131:43995
I20260811 12:07:34.705497 4891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.131:43995 every 8 connection(s)
I20260811 12:07:34.706777 4683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
I20260811 12:07:34.708403 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 4683
I20260811 12:07:34.708631 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance
I20260811 12:07:34.709214 10102 external_mini_cluster.cc:1428] Setting key 90207e323851b782a0eb739fc17f52ab
Aug 11 12:07:34 dist-test-slave-mjjr krb5kdc[4103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450054, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.131@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:07:34.726744 4897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:34.709645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:51023 (local address 127.9.221.190:35693)
0811 12:07:34.710128 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:34.710135 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:34.711288 (+ 1153us) server_negotiation.cc:407] Connection header received
0811 12:07:34.713222 (+ 1934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:34.713226 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:34.713290 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:34.713378 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:34.714772 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:34.715553 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:34.717084 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:34.717468 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:34.717558 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:34.720915 (+ 3357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:34.720920 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:34.720923 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:07:34.720963 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:34.723749 (+ 2786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:34.724658 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:34.724663 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:34.724666 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:34.724841 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:34.725248 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:34.725252 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:34.725254 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:34.725524 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:34.725643 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:34.726289 (+ 646us) server_negotiation.cc:299] Negotiation successful
0811 12:07:34.726455 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":224,"threads_started":1}
I20260811 12:07:34.727859 4902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:34.710295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:35693 (local address 127.9.221.131:51023)
0811 12:07:34.711126 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:34.711180 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 12:07:34.712968 (+ 1788us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:07:34.713584 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:07:34.713592 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:07:34.714074 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:07:34.714628 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:34.714644 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:34.715742 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:34.715747 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:34.716961 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:34.716973 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:34.717654 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:34.717658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:34.717844 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:34.718533 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:07:34.718561 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:07:34.720732 (+ 2171us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:07:34.723909 (+ 3177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:07:34.723915 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:07:34.723927 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:07:34.724520 (+ 593us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:07:34.724959 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:07:34.724962 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:07:34.724964 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:07:34.725145 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:07:34.725628 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:07:34.725634 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:07:34.725961 (+ 327us) client_negotiation.cc:769] Sending connection context
0811 12:07:34.726186 (+ 225us) client_negotiation.cc:240] Negotiation successful
0811 12:07:34.726577 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":354,"threads_started":1}
I20260811 12:07:34.729138 4892 heartbeater.cc:347] Connected to a master server at 127.9.221.190:35693
I20260811 12:07:34.729453 4892 heartbeater.cc:464] Registering TS with master...
I20260811 12:07:34.730175 4892 heartbeater.cc:511] Master 127.9.221.190:35693 requested a full tablet report, sending...
I20260811 12:07:34.731555 4171 ts_manager.cc:194] Registered new tserver with Master: a64bb7876be44a6e8c695478cbd115d8 (127.9.221.131:43995)
I20260811 12:07:34.732407 4171 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.131@KRBTEST.COM'} at 127.9.221.131:51023
I20260811 12:07:34.737183 10102 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:07:34.755192 4897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:34.741923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36902 (local address 127.9.221.190:35693)
0811 12:07:34.742153 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:34.742163 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:34.742570 (+ 407us) server_negotiation.cc:407] Connection header received
0811 12:07:34.742740 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:34.742747 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:34.742850 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:34.742982 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:34.744566 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:34.745300 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:34.746610 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:34.747068 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:34.747174 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:34.749699 (+ 2525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:34.749704 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:34.749707 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:07:34.749738 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:34.752543 (+ 2805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:34.753241 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:34.753245 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:34.753260 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:34.753379 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:34.753786 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:34.753790 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:34.753792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:34.754153 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:34.754356 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:34.754706 (+ 350us) server_negotiation.cc:299] Negotiation successful
0811 12:07:34.754929 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 12:07:34.759310 4171 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36902:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:07:34.766286 4171 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:07:34.789572 4920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:34.779508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49468 (local address 127.9.221.129:38297)
0811 12:07:34.780667 (+ 1159us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:34.780680 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:34.781523 (+ 843us) server_negotiation.cc:407] Connection header received
0811 12:07:34.781652 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:34.781664 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:34.781982 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:34.782201 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:07:34.783167 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:34.784863 (+ 1696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:34.787147 (+ 2284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:34.788424 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:34.788580 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:34.788825 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:34.789080 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 12:07:34.789240 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":934,"thread_start_us":730,"threads_started":1}
I20260811 12:07:34.789649 4919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:34.778810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:38297 (local address 127.0.0.1:49468)
0811 12:07:34.781190 (+ 2380us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:34.781213 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:07:34.781356 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:07:34.782561 (+ 1205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:07:34.782569 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:07:34.782600 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:07:34.782969 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:34.782983 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:34.785009 (+ 2026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:34.785014 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:34.786918 (+ 1904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:34.786933 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:34.788644 (+ 1711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:34.788650 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:34.788832 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:34.788897 (+ 65us) client_negotiation.cc:769] Sending connection context
0811 12:07:34.789026 (+ 129us) client_negotiation.cc:240] Negotiation successful
0811 12:07:34.789207 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2226,"mutex_wait_us":4,"thread_start_us":2033,"threads_started":1}
I20260811 12:07:34.791442 4923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:34.780604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.131:43995 (local address 127.0.0.1:33212)
0811 12:07:34.781803 (+ 1199us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:34.781826 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:07:34.781964 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:07:34.783165 (+ 1201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:07:34.783169 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:07:34.783194 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:07:34.783664 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:34.783680 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:34.785817 (+ 2137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:34.785826 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:34.788470 (+ 2644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:34.788480 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:34.790416 (+ 1936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:34.790424 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:34.790706 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:34.790737 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 12:07:34.790809 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 12:07:34.790905 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1053,"mutex_wait_us":249,"thread_start_us":917,"threads_started":1}
I20260811 12:07:34.791478 4917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:34.778021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.130:36191 (local address 127.0.0.1:35188)
0811 12:07:34.779447 (+ 1426us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:34.779504 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 12:07:34.779749 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:07:34.783474 (+ 3725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:07:34.783483 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:07:34.783511 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:07:34.784028 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:34.784041 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:34.785754 (+ 1713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:34.785760 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:34.788463 (+ 2703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:34.788479 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:34.790352 (+ 1873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:34.790359 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:34.790559 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:34.790698 (+ 139us) client_negotiation.cc:769] Sending connection context
0811 12:07:34.790771 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 12:07:34.790903 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1117,"mutex_wait_us":81,"thread_start_us":929,"threads_started":1}
I20260811 12:07:34.792388 4922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:34.780169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33212 (local address 127.9.221.131:43995)
0811 12:07:34.782597 (+ 2428us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:34.782605 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:34.782637 (+ 32us) server_negotiation.cc:407] Connection header received
0811 12:07:34.782732 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:34.782741 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:34.782964 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:34.783145 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:07:34.784067 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:34.785675 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:34.788663 (+ 2988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:34.790087 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:34.790230 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:34.790466 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:34.791881 (+ 1415us) server_negotiation.cc:299] Negotiation successful
0811 12:07:34.792041 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2258,"thread_start_us":2088,"threads_started":1}
I20260811 12:07:34.792395 4918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:34.778314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35188 (local address 127.9.221.130:36191)
0811 12:07:34.782923 (+ 4609us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:34.782929 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:34.782949 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:07:34.783018 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:34.783022 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:34.783248 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:34.784218 (+ 970us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:07:34.784430 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:34.785611 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:34.788659 (+ 3048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:34.790088 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:34.791622 (+ 1534us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:34.791854 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:34.791915 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 12:07:34.792056 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4473,"thread_start_us":4326,"threads_started":1}
I20260811 12:07:34.793903 4360 tablet_service.cc:1467] Processing CreateTablet for tablet ac65fc3328d947b6a218c6455a4b4a80 (DEFAULT_TABLE table=test-table [id=b94d2bd4e9bf4fe692f145002f96246e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:07:34.795207 4360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac65fc3328d947b6a218c6455a4b4a80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:07:34.795980 4792 tablet_service.cc:1467] Processing CreateTablet for tablet ac65fc3328d947b6a218c6455a4b4a80 (DEFAULT_TABLE table=test-table [id=b94d2bd4e9bf4fe692f145002f96246e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:07:34.796691 4582 tablet_service.cc:1467] Processing CreateTablet for tablet ac65fc3328d947b6a218c6455a4b4a80 (DEFAULT_TABLE table=test-table [id=b94d2bd4e9bf4fe692f145002f96246e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:07:34.797447 4792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac65fc3328d947b6a218c6455a4b4a80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:07:34.797994 4582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac65fc3328d947b6a218c6455a4b4a80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:07:34.806756 4924 tablet_bootstrap.cc:492] T ac65fc3328d947b6a218c6455a4b4a80 P 3e7bbd43a28845b0acd429692a7e27bb: Bootstrap starting.
I20260811 12:07:34.811013 4924 tablet_bootstrap.cc:654] T ac65fc3328d947b6a218c6455a4b4a80 P 3e7bbd43a28845b0acd429692a7e27bb: Neither blocks nor log segments found. Creating new log.
I20260811 12:07:34.812558 4928 tablet_bootstrap.cc:492] T ac65fc3328d947b6a218c6455a4b4a80 P a9c5164dba7449cabf766a1d5ed1f2df: Bootstrap starting.
I20260811 12:07:34.812544 4924 log.cc:824] T ac65fc3328d947b6a218c6455a4b4a80 P 3e7bbd43a28845b0acd429692a7e27bb: Log is configured to *not* fsync() on all Append() calls
I20260811 12:07:34.814307 4929 tablet_bootstrap.cc:492] T ac65fc3328d947b6a218c6455a4b4a80 P a64bb7876be44a6e8c695478cbd115d8: Bootstrap starting.
I20260811 12:07:34.816501 4928 tablet_bootstrap.cc:654] T ac65fc3328d947b6a218c6455a4b4a80 P a9c5164dba7449cabf766a1d5ed1f2df: Neither blocks nor log segments found. Creating new log.
I20260811 12:07:34.817740 4929 tablet_bootstrap.cc:654] T ac65fc3328d947b6a218c6455a4b4a80 P a64bb7876be44a6e8c695478cbd115d8: Neither blocks nor log segments found. Creating new log.
I20260811 12:07:34.817830 4928 log.cc:824] T ac65fc3328d947b6a218c6455a4b4a80 P a9c5164dba7449cabf766a1d5ed1f2df: Log is configured to *not* fsync() on all Append() calls
I20260811 12:07:34.817998 4924 tablet_bootstrap.cc:492] T ac65fc3328d947b6a218c6455a4b4a80 P 3e7bbd43a28845b0acd429692a7e27bb: No bootstrap required, opened a new log
I20260811 12:07:34.818320 4924 ts_tablet_manager.cc:1397] T ac65fc3328d947b6a218c6455a4b4a80 P 3e7bbd43a28845b0acd429692a7e27bb: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260811 12:07:34.819015 4929 log.cc:824] T ac65fc3328d947b6a218c6455a4b4a80 P a64bb7876be44a6e8c695478cbd115d8: Log is configured to *not* fsync() on all Append() calls
I20260811 12:07:34.822206 4928 tablet_bootstrap.cc:492] T ac65fc3328d947b6a218c6455a4b4a80 P a9c5164dba7449cabf766a1d5ed1f2df: No bootstrap required, opened a new log
I20260811 12:07:34.822410 4929 tablet_bootstrap.cc:492] T ac65fc3328d947b6a218c6455a4b4a80 P a64bb7876be44a6e8c695478cbd115d8: No bootstrap required, opened a new log
I20260811 12:07:34.822624 4928 ts_tablet_manager.cc:1397] T ac65fc3328d947b6a218c6455a4b4a80 P a9c5164dba7449cabf766a1d5ed1f2df: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260811 12:07:34.822680 4929 ts_tablet_manager.cc:1397] T ac65fc3328d947b6a218c6455a4b4a80 P a64bb7876be44a6e8c695478cbd115d8: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260811 12:07:34.829336 4924 raft_consensus.cc:348] T ac65fc3328d947b6a218c6455a4b4a80 P 3e7bbd43a28845b0acd429692a7e27bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9c5164dba7449cabf766a1d5ed1f2df" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 36191 } } peers { permanent_uuid: "a64bb7876be44a6e8c695478cbd115d8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43995 } } peers { permanent_uuid: "3e7bbd43a28845b0acd429692a7e27bb" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38297 } }
I20260811 12:07:34.829797 4924 raft_consensus.cc:374] T ac65fc3328d947b6a218c6455a4b4a80 P 3e7bbd43a28845b0acd429692a7e27bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:07:34.829905 4924 raft_consensus.cc:729] T ac65fc3328d947b6a218c6455a4b4a80 P 3e7bbd43a28845b0acd429692a7e27bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e7bbd43a28845b0acd429692a7e27bb, State: Initialized, Role: FOLLOWER
I20260811 12:07:34.830097 4928 raft_consensus.cc:348] T ac65fc3328d947b6a218c6455a4b4a80 P a9c5164dba7449cabf766a1d5ed1f2df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9c5164dba7449cabf766a1d5ed1f2df" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 36191 } } peers { permanent_uuid: "a64bb7876be44a6e8c695478cbd115d8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43995 } } peers { permanent_uuid: "3e7bbd43a28845b0acd429692a7e27bb" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38297 } }
I20260811 12:07:34.830523 4928 raft_consensus.cc:374] T ac65fc3328d947b6a218c6455a4b4a80 P a9c5164dba7449cabf766a1d5ed1f2df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:07:34.830621 4928 raft_consensus.cc:729] T ac65fc3328d947b6a218c6455a4b4a80 P a9c5164dba7449cabf766a1d5ed1f2df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9c5164dba7449cabf766a1d5ed1f2df, State: Initialized, Role: FOLLOWER
I20260811 12:07:34.830639 4924 consensus_queue.cc:260] T ac65fc3328d947b6a218c6455a4b4a80 P 3e7bbd43a28845b0acd429692a7e27bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9c5164dba7449cabf766a1d5ed1f2df" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 36191 } } peers { permanent_uuid: "a64bb7876be44a6e8c695478cbd115d8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43995 } } peers { permanent_uuid: "3e7bbd43a28845b0acd429692a7e27bb" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38297 } }
I20260811 12:07:34.831218 4928 consensus_queue.cc:260] T ac65fc3328d947b6a218c6455a4b4a80 P a9c5164dba7449cabf766a1d5ed1f2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9c5164dba7449cabf766a1d5ed1f2df" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 36191 } } peers { permanent_uuid: "a64bb7876be44a6e8c695478cbd115d8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43995 } } peers { permanent_uuid: "3e7bbd43a28845b0acd429692a7e27bb" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38297 } }
I20260811 12:07:34.832509 4676 heartbeater.cc:503] Master 127.9.221.190:35693 was elected leader, sending a full tablet report...
I20260811 12:07:34.832599 4924 ts_tablet_manager.cc:1428] T ac65fc3328d947b6a218c6455a4b4a80 P 3e7bbd43a28845b0acd429692a7e27bb: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260811 12:07:34.833092 4928 ts_tablet_manager.cc:1428] T ac65fc3328d947b6a218c6455a4b4a80 P a9c5164dba7449cabf766a1d5ed1f2df: Time spent starting tablet: real 0.010s user 0.011s sys 0.000s
I20260811 12:07:34.834398 4929 raft_consensus.cc:348] T ac65fc3328d947b6a218c6455a4b4a80 P a64bb7876be44a6e8c695478cbd115d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9c5164dba7449cabf766a1d5ed1f2df" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 36191 } } peers { permanent_uuid: "a64bb7876be44a6e8c695478cbd115d8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43995 } } peers { permanent_uuid: "3e7bbd43a28845b0acd429692a7e27bb" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38297 } }
I20260811 12:07:34.834818 4929 raft_consensus.cc:374] T ac65fc3328d947b6a218c6455a4b4a80 P a64bb7876be44a6e8c695478cbd115d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:07:34.834949 4929 raft_consensus.cc:729] T ac65fc3328d947b6a218c6455a4b4a80 P a64bb7876be44a6e8c695478cbd115d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a64bb7876be44a6e8c695478cbd115d8, State: Initialized, Role: FOLLOWER
I20260811 12:07:34.835276 4936 raft_consensus.cc:482] T ac65fc3328d947b6a218c6455a4b4a80 P a9c5164dba7449cabf766a1d5ed1f2df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:07:34.835714 4929 consensus_queue.cc:260] T ac65fc3328d947b6a218c6455a4b4a80 P a64bb7876be44a6e8c695478cbd115d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9c5164dba7449cabf766a1d5ed1f2df" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 36191 } } peers { permanent_uuid: "a64bb7876be44a6e8c695478cbd115d8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43995 } } peers { permanent_uuid: "3e7bbd43a28845b0acd429692a7e27bb" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38297 } }
I20260811 12:07:34.836947 4892 heartbeater.cc:503] Master 127.9.221.190:35693 was elected leader, sending a full tablet report...
I20260811 12:07:34.837420 4929 ts_tablet_manager.cc:1428] T ac65fc3328d947b6a218c6455a4b4a80 P a64bb7876be44a6e8c695478cbd115d8: Time spent starting tablet: real 0.015s user 0.007s sys 0.008s
I20260811 12:07:34.835795 4936 raft_consensus.cc:504] T ac65fc3328d947b6a218c6455a4b4a80 P a9c5164dba7449cabf766a1d5ed1f2df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9c5164dba7449cabf766a1d5ed1f2df" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 36191 } } peers { permanent_uuid: "a64bb7876be44a6e8c695478cbd115d8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43995 } } peers { permanent_uuid: "3e7bbd43a28845b0acd429692a7e27bb" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38297 } }
I20260811 12:07:34.840113 4936 leader_election.cc:288] T ac65fc3328d947b6a218c6455a4b4a80 P a9c5164dba7449cabf766a1d5ed1f2df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a64bb7876be44a6e8c695478cbd115d8 (127.9.221.131:43995), 3e7bbd43a28845b0acd429692a7e27bb (127.9.221.129:38297)
I20260811 12:07:34.848541 4922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:34.841047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:43879 (local address 127.9.221.131:43995)
0811 12:07:34.841247 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:34.841255 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:34.841576 (+ 321us) server_negotiation.cc:407] Connection header received
0811 12:07:34.841893 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:34.841901 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:34.841995 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:34.842073 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:07:34.843036 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:34.844185 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:34.846588 (+ 2403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:34.847699 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:34.847869 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:34.847958 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:34.848157 (+ 199us) server_negotiation.cc:299] Negotiation successful
0811 12:07:34.848286 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 12:07:34.848645 4940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:34.840438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.131:43995 (local address 127.9.221.130:43879)
0811 12:07:34.841468 (+ 1030us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:34.841491 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:07:34.841712 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:07:34.842359 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:07:34.842367 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:07:34.842397 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:07:34.842857 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:34.842872 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:34.844303 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:34.844319 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:34.846404 (+ 2085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:34.846421 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:34.847897 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:34.847901 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:34.848007 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:34.848030 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 12:07:34.848106 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 12:07:34.848195 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":874,"mutex_wait_us":1,"thread_start_us":1965,"threads_started":1}
I20260811 12:07:34.849465 4920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:34.840786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:55777 (local address 127.9.221.129:38297)
0811 12:07:34.841173 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:34.841180 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:34.842797 (+ 1617us) server_negotiation.cc:407] Connection header received
0811 12:07:34.843118 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:34.843125 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:34.843201 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:34.843288 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:07:34.844038 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:34.844723 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:34.846984 (+ 2261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:34.848234 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:34.848396 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:34.848465 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:34.849057 (+ 592us) server_negotiation.cc:299] Negotiation successful
0811 12:07:34.849190 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229}
I20260811 12:07:34.849699 4826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac65fc3328d947b6a218c6455a4b4a80" candidate_uuid: "a9c5164dba7449cabf766a1d5ed1f2df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a64bb7876be44a6e8c695478cbd115d8" is_pre_election: true
I20260811 12:07:34.850224 4826 raft_consensus.cc:2454] T ac65fc3328d947b6a218c6455a4b4a80 P a64bb7876be44a6e8c695478cbd115d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9c5164dba7449cabf766a1d5ed1f2df in term 0.
I20260811 12:07:34.850636 4939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:34.840784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:38297 (local address 127.9.221.130:55777)
0811 12:07:34.842698 (+ 1914us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:34.842718 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:07:34.842935 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:07:34.843460 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:07:34.843464 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:07:34.843488 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:07:34.843877 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:34.843887 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:34.844868 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:34.844874 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:34.846843 (+ 1969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:34.846853 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:34.848444 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:34.848450 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:34.848907 (+ 457us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:34.848934 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 12:07:34.849029 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 12:07:34.849151 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1768,"thread_start_us":400,"threads_started":1}
I20260811 12:07:34.850963 4506 leader_election.cc:302] T ac65fc3328d947b6a218c6455a4b4a80 P a9c5164dba7449cabf766a1d5ed1f2df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a64bb7876be44a6e8c695478cbd115d8, a9c5164dba7449cabf766a1d5ed1f2df; no voters:
I20260811 12:07:34.851387 4936 raft_consensus.cc:2781] T ac65fc3328d947b6a218c6455a4b4a80 P a9c5164dba7449cabf766a1d5ed1f2df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:07:34.851555 4936 raft_consensus.cc:482] T ac65fc3328d947b6a218c6455a4b4a80 P a9c5164dba7449cabf766a1d5ed1f2df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:07:34.851691 4936 raft_consensus.cc:3037] T ac65fc3328d947b6a218c6455a4b4a80 P a9c5164dba7449cabf766a1d5ed1f2df [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:07:34.851781 4396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac65fc3328d947b6a218c6455a4b4a80" candidate_uuid: "a9c5164dba7449cabf766a1d5ed1f2df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e7bbd43a28845b0acd429692a7e27bb" is_pre_election: true
I20260811 12:07:34.852339 4396 raft_consensus.cc:2454] T ac65fc3328d947b6a218c6455a4b4a80 P 3e7bbd43a28845b0acd429692a7e27bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9c5164dba7449cabf766a1d5ed1f2df in term 0.
I20260811 12:07:34.854521 4936 raft_consensus.cc:504] T ac65fc3328d947b6a218c6455a4b4a80 P a9c5164dba7449cabf766a1d5ed1f2df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9c5164dba7449cabf766a1d5ed1f2df" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 36191 } } peers { permanent_uuid: "a64bb7876be44a6e8c695478cbd115d8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43995 } } peers { permanent_uuid: "3e7bbd43a28845b0acd429692a7e27bb" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38297 } }
I20260811 12:07:34.855126 4936 leader_election.cc:288] T ac65fc3328d947b6a218c6455a4b4a80 P a9c5164dba7449cabf766a1d5ed1f2df [CANDIDATE]: Term 1 election: Requested vote from peers a64bb7876be44a6e8c695478cbd115d8 (127.9.221.131:43995), 3e7bbd43a28845b0acd429692a7e27bb (127.9.221.129:38297)
I20260811 12:07:34.855533 4826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac65fc3328d947b6a218c6455a4b4a80" candidate_uuid: "a9c5164dba7449cabf766a1d5ed1f2df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a64bb7876be44a6e8c695478cbd115d8"
I20260811 12:07:34.855614 4396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac65fc3328d947b6a218c6455a4b4a80" candidate_uuid: "a9c5164dba7449cabf766a1d5ed1f2df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e7bbd43a28845b0acd429692a7e27bb"
I20260811 12:07:34.855762 4826 raft_consensus.cc:3037] T ac65fc3328d947b6a218c6455a4b4a80 P a64bb7876be44a6e8c695478cbd115d8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:07:34.855849 4396 raft_consensus.cc:3037] T ac65fc3328d947b6a218c6455a4b4a80 P 3e7bbd43a28845b0acd429692a7e27bb [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:07:34.859045 4826 raft_consensus.cc:2454] T ac65fc3328d947b6a218c6455a4b4a80 P a64bb7876be44a6e8c695478cbd115d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9c5164dba7449cabf766a1d5ed1f2df in term 1.
I20260811 12:07:34.859045 4396 raft_consensus.cc:2454] T ac65fc3328d947b6a218c6455a4b4a80 P 3e7bbd43a28845b0acd429692a7e27bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9c5164dba7449cabf766a1d5ed1f2df in term 1.
I20260811 12:07:34.859423 4507 leader_election.cc:302] T ac65fc3328d947b6a218c6455a4b4a80 P a9c5164dba7449cabf766a1d5ed1f2df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e7bbd43a28845b0acd429692a7e27bb, a9c5164dba7449cabf766a1d5ed1f2df; no voters:
I20260811 12:07:34.859889 4936 raft_consensus.cc:2781] T ac65fc3328d947b6a218c6455a4b4a80 P a9c5164dba7449cabf766a1d5ed1f2df [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:07:34.860060 4936 raft_consensus.cc:686] T ac65fc3328d947b6a218c6455a4b4a80 P a9c5164dba7449cabf766a1d5ed1f2df [term 1 LEADER]: Becoming Leader. State: Replica: a9c5164dba7449cabf766a1d5ed1f2df, State: Running, Role: LEADER
I20260811 12:07:34.860550 4936 consensus_queue.cc:237] T ac65fc3328d947b6a218c6455a4b4a80 P a9c5164dba7449cabf766a1d5ed1f2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9c5164dba7449cabf766a1d5ed1f2df" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 36191 } } peers { permanent_uuid: "a64bb7876be44a6e8c695478cbd115d8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43995 } } peers { permanent_uuid: "3e7bbd43a28845b0acd429692a7e27bb" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38297 } }
I20260811 12:07:34.864468 4170 catalog_manager.cc:5495] T ac65fc3328d947b6a218c6455a4b4a80 P a9c5164dba7449cabf766a1d5ed1f2df reported cstate change: term changed from 0 to 1, leader changed from <none> to a9c5164dba7449cabf766a1d5ed1f2df (127.9.221.130). New cstate: current_term: 1 leader_uuid: "a9c5164dba7449cabf766a1d5ed1f2df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9c5164dba7449cabf766a1d5ed1f2df" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 36191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a64bb7876be44a6e8c695478cbd115d8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e7bbd43a28845b0acd429692a7e27bb" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38297 } health_report { overall_health: UNKNOWN } } }
W20260811 12:07:34.885682 4476 tablet.cc:2369] T ac65fc3328d947b6a218c6455a4b4a80 P 3e7bbd43a28845b0acd429692a7e27bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:07:34.891536 4918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:34.885859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35198 (local address 127.9.221.130:36191)
0811 12:07:34.886069 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:34.886075 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:34.886222 (+ 147us) server_negotiation.cc:407] Connection header received
0811 12:07:34.886494 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:34.886499 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:34.886591 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:34.886744 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:07:34.887137 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:34.887907 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:34.889377 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:34.889783 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:34.889925 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:34.890421 (+ 496us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:07:34.890917 (+ 496us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:07:34.891074 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:34.891135 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 12:07:34.891217 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 12:07:34.899726 4396 raft_consensus.cc:1261] T ac65fc3328d947b6a218c6455a4b4a80 P 3e7bbd43a28845b0acd429692a7e27bb [term 1 FOLLOWER]: Refusing update from remote peer a9c5164dba7449cabf766a1d5ed1f2df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:07:34.899729 4826 raft_consensus.cc:1261] T ac65fc3328d947b6a218c6455a4b4a80 P a64bb7876be44a6e8c695478cbd115d8 [term 1 FOLLOWER]: Refusing update from remote peer a9c5164dba7449cabf766a1d5ed1f2df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:07:34.900766 4934 consensus_queue.cc:1035] T ac65fc3328d947b6a218c6455a4b4a80 P a9c5164dba7449cabf766a1d5ed1f2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e7bbd43a28845b0acd429692a7e27bb" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 38297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:07:34.901126 4936 consensus_queue.cc:1035] T ac65fc3328d947b6a218c6455a4b4a80 P a9c5164dba7449cabf766a1d5ed1f2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "a64bb7876be44a6e8c695478cbd115d8" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 43995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:07:34.912425 4945 mvcc.cc:205] Tried to move back new op lower bound from 7317299424858468352 to 7317299424712687616. Current Snapshot: MvccSnapshot[applied={T|T < 7317299424858468352}]
I20260811 12:07:34.927707 4170 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36902:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:07:34.928022 4170 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36902:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:07:34.931831 4170 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 34e1530cf9064b18967c96f3fc145ae9: Sending DeleteTablet for 3 replicas of tablet ac65fc3328d947b6a218c6455a4b4a80
I20260811 12:07:34.933081 4792 tablet_service.cc:1514] Processing DeleteTablet for tablet ac65fc3328d947b6a218c6455a4b4a80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:07:34 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:33212
I20260811 12:07:34.933081 4582 tablet_service.cc:1514] Processing DeleteTablet for tablet ac65fc3328d947b6a218c6455a4b4a80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:07:34 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:35188
I20260811 12:07:34.933125 4360 tablet_service.cc:1514] Processing DeleteTablet for tablet ac65fc3328d947b6a218c6455a4b4a80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:07:34 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:49468
I20260811 12:07:34.934482 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 4259
I20260811 12:07:34.944314 4960 tablet_replica.cc:330] T ac65fc3328d947b6a218c6455a4b4a80 P 3e7bbd43a28845b0acd429692a7e27bb: stopping tablet replica
I20260811 12:07:34.944941 4960 raft_consensus.cc:2229] T ac65fc3328d947b6a218c6455a4b4a80 P 3e7bbd43a28845b0acd429692a7e27bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:07:34.945609 4960 raft_consensus.cc:2258] T ac65fc3328d947b6a218c6455a4b4a80 P 3e7bbd43a28845b0acd429692a7e27bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:07:34.948436 4959 tablet_replica.cc:330] T ac65fc3328d947b6a218c6455a4b4a80 P a64bb7876be44a6e8c695478cbd115d8: stopping tablet replica
I20260811 12:07:34.949141 4958 tablet_replica.cc:330] T ac65fc3328d947b6a218c6455a4b4a80 P a9c5164dba7449cabf766a1d5ed1f2df: stopping tablet replica
I20260811 12:07:34.949795 4960 ts_tablet_manager.cc:1905] T ac65fc3328d947b6a218c6455a4b4a80 P 3e7bbd43a28845b0acd429692a7e27bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:07:34.950007 4958 raft_consensus.cc:2229] T ac65fc3328d947b6a218c6455a4b4a80 P a9c5164dba7449cabf766a1d5ed1f2df [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:07:34.949647 4959 raft_consensus.cc:2229] T ac65fc3328d947b6a218c6455a4b4a80 P a64bb7876be44a6e8c695478cbd115d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:07:34.951004 4959 raft_consensus.cc:2258] T ac65fc3328d947b6a218c6455a4b4a80 P a64bb7876be44a6e8c695478cbd115d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:07:34.951718 4958 raft_consensus.cc:2258] T ac65fc3328d947b6a218c6455a4b4a80 P a9c5164dba7449cabf766a1d5ed1f2df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:07:34.953569 4959 ts_tablet_manager.cc:1905] T ac65fc3328d947b6a218c6455a4b4a80 P a64bb7876be44a6e8c695478cbd115d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:07:34.956875 4958 ts_tablet_manager.cc:1905] T ac65fc3328d947b6a218c6455a4b4a80 P a9c5164dba7449cabf766a1d5ed1f2df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:07:34.959264 4960 ts_tablet_manager.cc:1918] T ac65fc3328d947b6a218c6455a4b4a80 P 3e7bbd43a28845b0acd429692a7e27bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:07:34.959507 4960 log.cc:1196] T ac65fc3328d947b6a218c6455a4b4a80 P 3e7bbd43a28845b0acd429692a7e27bb: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-0/wal/wals/ac65fc3328d947b6a218c6455a4b4a80
I20260811 12:07:34.960188 4960 ts_tablet_manager.cc:1939] T ac65fc3328d947b6a218c6455a4b4a80 P 3e7bbd43a28845b0acd429692a7e27bb: Deleting consensus metadata
I20260811 12:07:34.960522 4959 ts_tablet_manager.cc:1918] T ac65fc3328d947b6a218c6455a4b4a80 P a64bb7876be44a6e8c695478cbd115d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:07:34.960682 4959 log.cc:1196] T ac65fc3328d947b6a218c6455a4b4a80 P a64bb7876be44a6e8c695478cbd115d8: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-2/wal/wals/ac65fc3328d947b6a218c6455a4b4a80
I20260811 12:07:34.961274 4959 ts_tablet_manager.cc:1939] T ac65fc3328d947b6a218c6455a4b4a80 P a64bb7876be44a6e8c695478cbd115d8: Deleting consensus metadata
I20260811 12:07:34.962544 4146 catalog_manager.cc:4841] TS 3e7bbd43a28845b0acd429692a7e27bb (127.9.221.129:38297): tablet ac65fc3328d947b6a218c6455a4b4a80 (table test-table [id=b94d2bd4e9bf4fe692f145002f96246e]) successfully deleted
I20260811 12:07:34.963601 4144 catalog_manager.cc:4841] TS a64bb7876be44a6e8c695478cbd115d8 (127.9.221.131:43995): tablet ac65fc3328d947b6a218c6455a4b4a80 (table test-table [id=b94d2bd4e9bf4fe692f145002f96246e]) successfully deleted
Aug 11 12:07:34 dist-test-slave-mjjr krb5kdc[4103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.129@KRBTEST.COM
I20260811 12:07:34.966709 4958 ts_tablet_manager.cc:1918] T ac65fc3328d947b6a218c6455a4b4a80 P a9c5164dba7449cabf766a1d5ed1f2df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:07:34.966881 4958 log.cc:1196] T ac65fc3328d947b6a218c6455a4b4a80 P a9c5164dba7449cabf766a1d5ed1f2df: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449743084027-10102-0/minicluster-data/ts-1/wal/wals/ac65fc3328d947b6a218c6455a4b4a80
I20260811 12:07:34.967469 4958 ts_tablet_manager.cc:1939] T ac65fc3328d947b6a218c6455a4b4a80 P a9c5164dba7449cabf766a1d5ed1f2df: Deleting consensus metadata
I20260811 12:07:34.969193 4144 catalog_manager.cc:4841] TS a9c5164dba7449cabf766a1d5ed1f2df (127.9.221.130:36191): tablet ac65fc3328d947b6a218c6455a4b4a80 (table test-table [id=b94d2bd4e9bf4fe692f145002f96246e]) successfully deleted
I20260811 12:07:34.972430 4920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:34.936282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49476 (local address 127.9.221.129:38297)
0811 12:07:34.936532 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:34.936541 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:34.946901 (+ 10360us) server_negotiation.cc:407] Connection header received
0811 12:07:34.947407 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:34.947415 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:34.947519 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:34.947613 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:34.949753 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:34.951041 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:34.954822 (+ 3781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:34.955558 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:34.955759 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:34.965297 (+ 9538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:34.965311 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:34.965329 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:07:34.965368 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:34.969364 (+ 3996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:34.970323 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:34.970330 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:34.970333 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:34.970468 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:34.970921 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:34.970924 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:34.970926 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:34.971236 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:34.971453 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:34.971903 (+ 450us) server_negotiation.cc:299] Negotiation successful
0811 12:07:34.972102 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 12:07:34.973099 4466 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:07:35.064960 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 4259
I20260811 12:07:35.091878 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 4483
Aug 11 12:07:35 dist-test-slave-mjjr krb5kdc[4103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.130@KRBTEST.COM
I20260811 12:07:35.109691 4918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:35.092658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35204 (local address 127.9.221.130:36191)
0811 12:07:35.092839 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:35.092843 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:35.092930 (+ 87us) server_negotiation.cc:407] Connection header received
0811 12:07:35.093105 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:35.093109 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:35.093176 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:35.093258 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:35.095010 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:35.095957 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:35.097918 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:35.098337 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:35.098435 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:35.103458 (+ 5023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:35.103471 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:35.103486 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:07:35.103527 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:35.106548 (+ 3021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:35.107323 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:35.107341 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:35.107343 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:35.107484 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:35.107918 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:35.107925 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:35.107929 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:35.108284 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:35.108648 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:35.109056 (+ 408us) server_negotiation.cc:299] Negotiation successful
0811 12:07:35.109282 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 12:07:35.110379 4670 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:07:35.222270 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 4483
I20260811 12:07:35.246544 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 4683
Aug 11 12:07:35 dist-test-slave-mjjr krb5kdc[4103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.131@KRBTEST.COM
I20260811 12:07:35.262085 4922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:35.247306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33214 (local address 127.9.221.131:43995)
0811 12:07:35.247500 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:35.247504 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:35.247523 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:07:35.247787 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:35.247797 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:35.247863 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:35.247946 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:35.249258 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:35.250082 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:35.251424 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:35.251938 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:35.252035 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:35.256493 (+ 4458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:35.256509 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:35.256524 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:07:35.256578 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:35.259409 (+ 2831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:35.260128 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:35.260136 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:35.260139 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:35.260323 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:35.260649 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:35.260652 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:35.260654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:35.261059 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:35.261317 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:35.261563 (+ 246us) server_negotiation.cc:299] Negotiation successful
0811 12:07:35.261726 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 12:07:35.262759 4884 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:07:35.353413 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 4683
I20260811 12:07:35.381045 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 4122
I20260811 12:07:35.382257 4212 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:07:35.494796 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 4122
2026-08-11T12:07:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4568 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-11T12:07:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:07:35Z Disabled control of system clock
I20260811 12:07:35.593542 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:33157
--webserver_interface=127.9.221.190
--webserver_port=0
--builtin_ntp_servers=127.9.221.148:40887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:33157
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 12:07:35.945680 4972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:07:35.946066 4972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:07:35.946231 4972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:07:35.953734 4972 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:07:35.953858 4972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:07:35.953905 4972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:07:35.953958 4972 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:07:35.954010 4972 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:07:35.963209 4972 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:40887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:33157
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:33157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.221.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:07:35.964534 4972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:07:35.966264 4972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:07:35.978251 4978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:35.978279 4979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:35.978543 4981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:07:35.979338 4972 server_base.cc:1034] running on GCE node
I20260811 12:07:35.980160 4972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:07:35.981575 4972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:07:35.982810 4972 hybrid_clock.cc:648] HybridClock initialized: now 1786450055982774 us; error 48 us; skew 500 ppm
I20260811 12:07:35.983167 4972 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:07:35.985983 4972 webserver.cc:460] Webserver started at http://127.9.221.190:45549/ using document root <none> and password file <none>
I20260811 12:07:35.986696 4972 fs_manager.cc:359] Metadata directory not provided
I20260811 12:07:35.986827 4972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:07:35.987201 4972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:07:35.990224 4972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "5715fbad9dd04b709d7fae42b1be8200"
format_stamp: "Formatted at 2026-08-11 12:07:35 on dist-test-slave-mjjr"
server_key: "6f5a0f32e004a9e314ae09e8e8835900"
server_key_iv: "c227cd8f5b8336322727b64dee9f0174"
server_key_version: "encryptionkey@0"
I20260811 12:07:35.991080 4972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "5715fbad9dd04b709d7fae42b1be8200"
format_stamp: "Formatted at 2026-08-11 12:07:35 on dist-test-slave-mjjr"
server_key: "6f5a0f32e004a9e314ae09e8e8835900"
server_key_iv: "c227cd8f5b8336322727b64dee9f0174"
server_key_version: "encryptionkey@0"
I20260811 12:07:35.996249 4972 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 12:07:35.999606 4990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:36.001016 4972 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:07:36.001211 4972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "5715fbad9dd04b709d7fae42b1be8200"
format_stamp: "Formatted at 2026-08-11 12:07:35 on dist-test-slave-mjjr"
server_key: "6f5a0f32e004a9e314ae09e8e8835900"
server_key_iv: "c227cd8f5b8336322727b64dee9f0174"
server_key_version: "encryptionkey@0"
I20260811 12:07:36.001439 4972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:07:36.016852 4972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:07:36.017679 4972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:07:36.017993 4972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:07:36.037173 4972 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.190:33157
I20260811 12:07:36.037204 5071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.190:33157 every 8 connection(s)
I20260811 12:07:36.038537 4972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:07:36.042253 5072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:07:36.042639 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 4972
I20260811 12:07:36.042837 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:07:36.043256 10102 external_mini_cluster.cc:1428] Setting key 45702518ca2e83c93e8423c2c2a9732a
I20260811 12:07:36.055809 5072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5715fbad9dd04b709d7fae42b1be8200: Bootstrap starting.
I20260811 12:07:36.058087 5078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:36.045183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39982 (local address 127.9.221.190:33157)
0811 12:07:36.046316 (+ 1133us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:36.046327 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:36.046370 (+ 43us) server_negotiation.cc:407] Connection header received
0811 12:07:36.048401 (+ 2031us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:36.048423 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:36.048865 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:36.049199 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:36.049788 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:36.051017 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:36.052986 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:36.053653 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:36.053789 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:36.055060 (+ 1271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:36.055068 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:36.055080 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:07:36.055606 (+ 526us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:36.055763 (+ 157us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:07:36.055795 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:36.055892 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:36.056459 (+ 567us) server_negotiation.cc:299] Negotiation successful
0811 12:07:36.056763 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":712,"thread_start_us":470,"threads_started":1}
I20260811 12:07:36.059567 5072 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5715fbad9dd04b709d7fae42b1be8200: Neither blocks nor log segments found. Creating new log.
I20260811 12:07:36.060614 5072 log.cc:824] T 00000000000000000000000000000000 P 5715fbad9dd04b709d7fae42b1be8200: Log is configured to *not* fsync() on all Append() calls
I20260811 12:07:36.063246 5072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5715fbad9dd04b709d7fae42b1be8200: No bootstrap required, opened a new log
I20260811 12:07:36.070351 5072 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5715fbad9dd04b709d7fae42b1be8200 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5715fbad9dd04b709d7fae42b1be8200" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 33157 } }
I20260811 12:07:36.070705 5072 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5715fbad9dd04b709d7fae42b1be8200 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:07:36.070809 5072 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5715fbad9dd04b709d7fae42b1be8200 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5715fbad9dd04b709d7fae42b1be8200, State: Initialized, Role: FOLLOWER
I20260811 12:07:36.071410 5072 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5715fbad9dd04b709d7fae42b1be8200 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5715fbad9dd04b709d7fae42b1be8200" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 33157 } }
I20260811 12:07:36.071682 5072 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5715fbad9dd04b709d7fae42b1be8200 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:07:36.071838 5072 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5715fbad9dd04b709d7fae42b1be8200 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:07:36.072019 5072 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5715fbad9dd04b709d7fae42b1be8200 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:07:36.074048 5072 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5715fbad9dd04b709d7fae42b1be8200 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5715fbad9dd04b709d7fae42b1be8200" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 33157 } }
I20260811 12:07:36.074519 5072 leader_election.cc:302] T 00000000000000000000000000000000 P 5715fbad9dd04b709d7fae42b1be8200 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5715fbad9dd04b709d7fae42b1be8200; no voters:
I20260811 12:07:36.075176 5072 leader_election.cc:288] T 00000000000000000000000000000000 P 5715fbad9dd04b709d7fae42b1be8200 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:07:36.075363 5080 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5715fbad9dd04b709d7fae42b1be8200 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:07:36.076145 5080 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5715fbad9dd04b709d7fae42b1be8200 [term 1 LEADER]: Becoming Leader. State: Replica: 5715fbad9dd04b709d7fae42b1be8200, State: Running, Role: LEADER
I20260811 12:07:36.076617 5080 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5715fbad9dd04b709d7fae42b1be8200 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5715fbad9dd04b709d7fae42b1be8200" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 33157 } }
I20260811 12:07:36.077358 5072 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5715fbad9dd04b709d7fae42b1be8200 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:07:36.079814 5081 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5715fbad9dd04b709d7fae42b1be8200 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5715fbad9dd04b709d7fae42b1be8200" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5715fbad9dd04b709d7fae42b1be8200" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 33157 } } }
I20260811 12:07:36.079697 5082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5715fbad9dd04b709d7fae42b1be8200 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5715fbad9dd04b709d7fae42b1be8200. Latest consensus state: current_term: 1 leader_uuid: "5715fbad9dd04b709d7fae42b1be8200" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5715fbad9dd04b709d7fae42b1be8200" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 33157 } } }
I20260811 12:07:36.080175 5081 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5715fbad9dd04b709d7fae42b1be8200 [sys.catalog]: This master's current role is: LEADER
I20260811 12:07:36.080215 5082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5715fbad9dd04b709d7fae42b1be8200 [sys.catalog]: This master's current role is: LEADER
I20260811 12:07:36.082917 5090 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:07:36.087973 5090 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:07:36.096627 5090 catalog_manager.cc:1347] Generated new cluster ID: 8b34ac433d93448d98c54e9d06573133
I20260811 12:07:36.096760 5090 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:07:36.106237 5090 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:07:36.107405 5090 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:07:36.125154 5090 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5715fbad9dd04b709d7fae42b1be8200: Generated new TSK 0
I20260811 12:07:36.126014 5090 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:07:36.139710 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.129:0
--local_ip_for_outbound_sockets=127.9.221.129
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:33157
--builtin_ntp_servers=127.9.221.148:40887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 12:07:36.504441 5112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:07:36.504756 5112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:07:36.504868 5112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:07:36.512221 5112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:07:36.512308 5112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:07:36.512415 5112 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.129
I20260811 12:07:36.521096 5112 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:40887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:33157
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.129
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:07:36.522292 5112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:07:36.524180 5112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:07:36.537639 5119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:36.537771 5122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:36.537699 5126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:07:36.538475 5112 server_base.cc:1034] running on GCE node
I20260811 12:07:36.539055 5112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:07:36.540045 5112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:07:36.541247 5112 hybrid_clock.cc:648] HybridClock initialized: now 1786450056541204 us; error 41 us; skew 500 ppm
I20260811 12:07:36.541555 5112 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:07:36.543702 5112 webserver.cc:460] Webserver started at http://127.9.221.129:36517/ using document root <none> and password file <none>
I20260811 12:07:36.544401 5112 fs_manager.cc:359] Metadata directory not provided
I20260811 12:07:36.544520 5112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:07:36.544853 5112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:07:36.547820 5112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-0/data/instance:
uuid: "9e4bec4edae943fba5fab62ae4851de3"
format_stamp: "Formatted at 2026-08-11 12:07:36 on dist-test-slave-mjjr"
server_key: "505037395c634be4a09fc07426383949"
server_key_iv: "a2e36ec841af2212a0af8e3d7f29c631"
server_key_version: "encryptionkey@0"
I20260811 12:07:36.548594 5112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance:
uuid: "9e4bec4edae943fba5fab62ae4851de3"
format_stamp: "Formatted at 2026-08-11 12:07:36 on dist-test-slave-mjjr"
server_key: "505037395c634be4a09fc07426383949"
server_key_iv: "a2e36ec841af2212a0af8e3d7f29c631"
server_key_version: "encryptionkey@0"
I20260811 12:07:36.553561 5112 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 12:07:36.556967 5135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:36.558311 5112 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:07:36.558470 5112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-0/wal
uuid: "9e4bec4edae943fba5fab62ae4851de3"
format_stamp: "Formatted at 2026-08-11 12:07:36 on dist-test-slave-mjjr"
server_key: "505037395c634be4a09fc07426383949"
server_key_iv: "a2e36ec841af2212a0af8e3d7f29c631"
server_key_version: "encryptionkey@0"
I20260811 12:07:36.558673 5112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:07:36.584649 5112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:07:36.585448 5112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:07:36.585744 5112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:07:36.586730 5112 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:07:36.588212 5112 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:07:36.588327 5112 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:36.588460 5112 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:07:36.588559 5112 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:36.622776 5112 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.129:43965
I20260811 12:07:36.622822 5317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.129:43965 every 8 connection(s)
I20260811 12:07:36.624208 5112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
I20260811 12:07:36.624823 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 5112
I20260811 12:07:36.625013 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance
I20260811 12:07:36.625672 10102 external_mini_cluster.cc:1428] Setting key 7a7a1d13764961ce8ab5ea5e0c121363
I20260811 12:07:36.629046 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.130:0
--local_ip_for_outbound_sockets=127.9.221.130
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:33157
--builtin_ntp_servers=127.9.221.148:40887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 12:07:36.637907 5323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:36.627035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:33125 (local address 127.9.221.190:33157)
0811 12:07:36.627985 (+ 950us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:36.627992 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:36.629142 (+ 1150us) server_negotiation.cc:407] Connection header received
0811 12:07:36.630915 (+ 1773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:36.630922 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:36.631011 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:36.631129 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:36.632213 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:36.633322 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:36.634823 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:36.635508 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:36.635674 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:36.636935 (+ 1261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:36.636943 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:36.636950 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:07:36.636974 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:36.637028 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:07:36.637046 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:36.637137 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:36.637359 (+ 222us) server_negotiation.cc:299] Negotiation successful
0811 12:07:36.637466 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":821,"thread_start_us":741,"threads_started":1}
I20260811 12:07:36.638605 5324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:36.627849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:33157 (local address 127.9.221.129:33125)
0811 12:07:36.628962 (+ 1113us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:36.629014 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 12:07:36.630651 (+ 1637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:07:36.631321 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:07:36.631328 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:07:36.631413 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:07:36.632042 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:36.632063 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:36.633473 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:36.633477 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:36.634674 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:36.634686 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:36.635755 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:36.635760 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:36.635956 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:36.636630 (+ 674us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:07:36.636632 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:07:36.636672 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:07:36.636738 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:07:36.637140 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:07:36.637148 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:07:36.637153 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 12:07:36.637281 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 12:07:36.637552 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":367,"threads_started":1}
I20260811 12:07:36.640115 5318 heartbeater.cc:347] Connected to a master server at 127.9.221.190:33157
I20260811 12:07:36.640413 5318 heartbeater.cc:464] Registering TS with master...
I20260811 12:07:36.641289 5318 heartbeater.cc:511] Master 127.9.221.190:33157 requested a full tablet report, sending...
I20260811 12:07:36.643584 5022 ts_manager.cc:194] Registered new tserver with Master: 9e4bec4edae943fba5fab62ae4851de3 (127.9.221.129:43965)
I20260811 12:07:36.645756 5022 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.221.129:33125
W20260811 12:07:36.967257 5327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:07:36.967535 5327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:07:36.967623 5327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:07:36.974766 5327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:07:36.974858 5327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:07:36.974961 5327 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.130
I20260811 12:07:36.983968 5327 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:40887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:33157
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.130
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:07:36.985169 5327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:07:36.986898 5327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:07:37.004158 5335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:37.004197 5336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:37.004122 5338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:07:37.005206 5327 server_base.cc:1034] running on GCE node
I20260811 12:07:37.005908 5327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:07:37.006919 5327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:07:37.008148 5327 hybrid_clock.cc:648] HybridClock initialized: now 1786450057008111 us; error 51 us; skew 500 ppm
I20260811 12:07:37.008471 5327 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:07:37.010782 5327 webserver.cc:460] Webserver started at http://127.9.221.130:35923/ using document root <none> and password file <none>
I20260811 12:07:37.011476 5327 fs_manager.cc:359] Metadata directory not provided
I20260811 12:07:37.011615 5327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:07:37.011986 5327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:07:37.014830 5327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-1/data/instance:
uuid: "7658f50444e74655a24292c521371a08"
format_stamp: "Formatted at 2026-08-11 12:07:37 on dist-test-slave-mjjr"
server_key: "c581ac71260a254e84fc8446394d1344"
server_key_iv: "466d11665e04230611330ba14fc52f25"
server_key_version: "encryptionkey@0"
I20260811 12:07:37.015619 5327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance:
uuid: "7658f50444e74655a24292c521371a08"
format_stamp: "Formatted at 2026-08-11 12:07:37 on dist-test-slave-mjjr"
server_key: "c581ac71260a254e84fc8446394d1344"
server_key_iv: "466d11665e04230611330ba14fc52f25"
server_key_version: "encryptionkey@0"
I20260811 12:07:37.020772 5327 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 12:07:37.024147 5345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:37.025418 5327 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 12:07:37.025593 5327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-1/wal
uuid: "7658f50444e74655a24292c521371a08"
format_stamp: "Formatted at 2026-08-11 12:07:37 on dist-test-slave-mjjr"
server_key: "c581ac71260a254e84fc8446394d1344"
server_key_iv: "466d11665e04230611330ba14fc52f25"
server_key_version: "encryptionkey@0"
I20260811 12:07:37.025839 5327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:07:37.040508 5327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:07:37.041404 5327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:07:37.041715 5327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:07:37.042824 5327 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:07:37.044543 5327 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:07:37.044646 5327 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:37.044742 5327 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:07:37.044822 5327 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:37.082052 5327 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.130:40071
I20260811 12:07:37.082100 5520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.130:40071 every 8 connection(s)
I20260811 12:07:37.083449 5327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
I20260811 12:07:37.091413 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 5327
I20260811 12:07:37.091612 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance
I20260811 12:07:37.092110 10102 external_mini_cluster.cc:1428] Setting key efab865b0c200f64aed6ae6c1367396e
I20260811 12:07:37.095306 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.131:0
--local_ip_for_outbound_sockets=127.9.221.131
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:33157
--builtin_ntp_servers=127.9.221.148:40887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 12:07:37.096026 5323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:37.086019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:49011 (local address 127.9.221.190:33157)
0811 12:07:37.086205 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:37.086210 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:37.087472 (+ 1262us) server_negotiation.cc:407] Connection header received
0811 12:07:37.089210 (+ 1738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:37.089218 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:37.089315 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:37.089429 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:37.090499 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:37.091225 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:37.093132 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:37.093574 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:37.093706 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:37.095254 (+ 1548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:37.095259 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:37.095263 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:07:37.095279 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:37.095313 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:07:37.095325 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:37.095399 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:37.095711 (+ 312us) server_negotiation.cc:299] Negotiation successful
0811 12:07:37.095796 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 12:07:37.097637 5524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:37.086432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:33157 (local address 127.9.221.130:49011)
0811 12:07:37.087320 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:37.087362 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 12:07:37.088925 (+ 1563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:07:37.089648 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:07:37.089656 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:07:37.089736 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:07:37.090325 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:37.090339 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:37.091384 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:37.091392 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:37.092987 (+ 1595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:37.092999 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:37.093832 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:37.093841 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:37.094115 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:37.094880 (+ 765us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:07:37.094885 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:07:37.094941 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:07:37.095055 (+ 114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:07:37.095433 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:07:37.095447 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:07:37.095457 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 12:07:37.095673 (+ 216us) client_negotiation.cc:240] Negotiation successful
0811 12:07:37.096165 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":278,"threads_started":1}
I20260811 12:07:37.099098 5522 heartbeater.cc:347] Connected to a master server at 127.9.221.190:33157
I20260811 12:07:37.099399 5522 heartbeater.cc:464] Registering TS with master...
I20260811 12:07:37.100162 5522 heartbeater.cc:511] Master 127.9.221.190:33157 requested a full tablet report, sending...
I20260811 12:07:37.101610 5022 ts_manager.cc:194] Registered new tserver with Master: 7658f50444e74655a24292c521371a08 (127.9.221.130:40071)
I20260811 12:07:37.102415 5022 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.221.130:49011
W20260811 12:07:37.435528 5525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:07:37.435828 5525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:07:37.435993 5525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:07:37.443138 5525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:07:37.443231 5525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:07:37.443449 5525 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.131
I20260811 12:07:37.452951 5525 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:40887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:33157
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.131
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:07:37.454308 5525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:07:37.456223 5525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:07:37.469889 5535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:37.469995 5540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:37.469964 5536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:07:37.470805 5525 server_base.cc:1034] running on GCE node
I20260811 12:07:37.471376 5525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:07:37.472461 5525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:07:37.473681 5525 hybrid_clock.cc:648] HybridClock initialized: now 1786450057473630 us; error 47 us; skew 500 ppm
I20260811 12:07:37.474079 5525 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:07:37.476558 5525 webserver.cc:460] Webserver started at http://127.9.221.131:43249/ using document root <none> and password file <none>
I20260811 12:07:37.477314 5525 fs_manager.cc:359] Metadata directory not provided
I20260811 12:07:37.477415 5525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:07:37.477797 5525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:07:37.480822 5525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-2/data/instance:
uuid: "24df44299c104ccd8c09590a86f86001"
format_stamp: "Formatted at 2026-08-11 12:07:37 on dist-test-slave-mjjr"
server_key: "7bb3d29650e9f084f6ff15cd8c52ce18"
server_key_iv: "649ef8cc1019f36a1b33e549d466eb74"
server_key_version: "encryptionkey@0"
I20260811 12:07:37.481711 5525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance:
uuid: "24df44299c104ccd8c09590a86f86001"
format_stamp: "Formatted at 2026-08-11 12:07:37 on dist-test-slave-mjjr"
server_key: "7bb3d29650e9f084f6ff15cd8c52ce18"
server_key_iv: "649ef8cc1019f36a1b33e549d466eb74"
server_key_version: "encryptionkey@0"
I20260811 12:07:37.487390 5525 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260811 12:07:37.492679 5549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:37.494102 5525 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260811 12:07:37.494424 5525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-2/wal
uuid: "24df44299c104ccd8c09590a86f86001"
format_stamp: "Formatted at 2026-08-11 12:07:37 on dist-test-slave-mjjr"
server_key: "7bb3d29650e9f084f6ff15cd8c52ce18"
server_key_iv: "649ef8cc1019f36a1b33e549d466eb74"
server_key_version: "encryptionkey@0"
I20260811 12:07:37.494673 5525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:07:37.515457 5525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:07:37.516342 5525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:07:37.516669 5525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:07:37.517678 5525 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:07:37.519179 5525 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:07:37.519307 5525 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:37.519454 5525 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:07:37.519546 5525 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:37.554304 5525 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.131:35415
I20260811 12:07:37.554363 5717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.131:35415 every 8 connection(s)
I20260811 12:07:37.555915 5525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
I20260811 12:07:37.556409 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 5525
I20260811 12:07:37.556638 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance
I20260811 12:07:37.557238 10102 external_mini_cluster.cc:1428] Setting key 5199f8bc7ac3daaedcd53fe7a678e432
I20260811 12:07:37.568828 5323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:37.558776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:35073 (local address 127.9.221.190:33157)
0811 12:07:37.558966 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:37.558970 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:37.560700 (+ 1730us) server_negotiation.cc:407] Connection header received
0811 12:07:37.562470 (+ 1770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:37.562476 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:37.562567 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:37.562670 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:37.563811 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:37.564571 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:37.566063 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:37.566460 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:37.566605 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:37.567965 (+ 1360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:37.567970 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:37.567973 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:07:37.567989 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:37.568019 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:07:37.568030 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:37.568128 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:37.568368 (+ 240us) server_negotiation.cc:299] Negotiation successful
0811 12:07:37.568496 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 12:07:37.569748 5720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:37.559471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:33157 (local address 127.9.221.131:35073)
0811 12:07:37.560531 (+ 1060us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:37.560582 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 12:07:37.562215 (+ 1633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:07:37.562911 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:07:37.562918 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:07:37.562994 (+ 76us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:07:37.563609 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:37.563627 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:37.564717 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:37.564723 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:37.565924 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:37.565936 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:37.566690 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:37.566697 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:37.566919 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:37.567578 (+ 659us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:07:37.567580 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:07:37.567616 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:07:37.567760 (+ 144us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:07:37.568129 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:07:37.568143 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:07:37.568153 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 12:07:37.568333 (+ 180us) client_negotiation.cc:240] Negotiation successful
0811 12:07:37.568703 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":351,"threads_started":1}
I20260811 12:07:37.571219 5718 heartbeater.cc:347] Connected to a master server at 127.9.221.190:33157
I20260811 12:07:37.571561 5718 heartbeater.cc:464] Registering TS with master...
I20260811 12:07:37.572540 5718 heartbeater.cc:511] Master 127.9.221.190:33157 requested a full tablet report, sending...
I20260811 12:07:37.574067 5022 ts_manager.cc:194] Registered new tserver with Master: 24df44299c104ccd8c09590a86f86001 (127.9.221.131:35415)
I20260811 12:07:37.574963 5022 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.221.131:35073
I20260811 12:07:37.585122 10102 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:07:37.595465 5323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:37.589613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39992 (local address 127.9.221.190:33157)
0811 12:07:37.589781 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:37.589785 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:37.590306 (+ 521us) server_negotiation.cc:407] Connection header received
0811 12:07:37.590452 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:37.590455 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:37.590523 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:37.590602 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:37.591100 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:37.591845 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:37.593230 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:37.593643 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:37.593738 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:37.594735 (+ 997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:37.594740 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:37.594743 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:07:37.594759 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:37.594790 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:07:37.594801 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:37.594912 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:37.595082 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 12:07:37.595168 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 12:07:37.599346 5022 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39992:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:07:37.606307 5022 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:07:37.630700 5737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:37.618510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:43965 (local address 127.0.0.1:48476)
0811 12:07:37.619632 (+ 1122us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:37.619717 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0811 12:07:37.619962 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:07:37.622668 (+ 2706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:07:37.622675 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:07:37.622707 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:07:37.623154 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:37.623169 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:37.625257 (+ 2088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:37.625263 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:37.627841 (+ 2578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:37.627854 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:37.629773 (+ 1919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:37.629779 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:37.629986 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:37.630043 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 12:07:37.630157 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 12:07:37.630359 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":782,"thread_start_us":594,"threads_started":1}
I20260811 12:07:37.631543 5741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:37.618711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48476 (local address 127.9.221.129:43965)
0811 12:07:37.621769 (+ 3058us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:37.621782 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:37.621819 (+ 37us) server_negotiation.cc:407] Connection header received
0811 12:07:37.621938 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:37.621945 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:37.622183 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:37.622405 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:07:37.623378 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:37.625073 (+ 1695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:37.628073 (+ 3000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:37.629523 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:37.630315 (+ 792us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:37.630590 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:37.630664 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 12:07:37.631132 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2863,"thread_start_us":2741,"threads_started":1}
I20260811 12:07:37.635699 5740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:37.618706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.131:35415 (local address 127.0.0.1:47922)
0811 12:07:37.625018 (+ 6312us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:37.625044 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 12:07:37.625710 (+ 666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:07:37.626940 (+ 1230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:07:37.626944 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:07:37.626968 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:07:37.627420 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:37.627434 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:37.630640 (+ 3206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:37.630649 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:37.632978 (+ 2329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:37.632994 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:37.634829 (+ 1835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:37.634836 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:37.635040 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:37.635076 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 12:07:37.635167 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 12:07:37.635293 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6123,"mutex_wait_us":1,"thread_start_us":8994,"threads_started":1}
I20260811 12:07:37.635820 5743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:37.619181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47922 (local address 127.9.221.131:35415)
0811 12:07:37.621067 (+ 1886us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:37.621080 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:37.625167 (+ 4087us) server_negotiation.cc:407] Connection header received
0811 12:07:37.626093 (+ 926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:37.626106 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:37.626404 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:37.626589 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:07:37.628352 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:37.630389 (+ 2037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:37.633190 (+ 2801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:37.634585 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:37.634726 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:37.635017 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:37.635260 (+ 243us) server_negotiation.cc:299] Negotiation successful
0811 12:07:37.635435 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1661,"thread_start_us":1513,"threads_started":1}
I20260811 12:07:37.636085 5217 tablet_service.cc:1467] Processing CreateTablet for tablet b0e4a86d768b408185f29f9a8ed55f44 (DEFAULT_TABLE table=test-table [id=99735be780c54fb1b91090122fe1bb57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:07:37.637743 5217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0e4a86d768b408185f29f9a8ed55f44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:07:37.638475 5737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:37.621889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.130:40071 (local address 127.0.0.1:49160)
0811 12:07:37.630992 (+ 9103us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:37.631010 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:07:37.631106 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:07:37.632562 (+ 1456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:07:37.632569 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:07:37.632600 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:07:37.632738 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:37.632747 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:37.634245 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:37.634250 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:37.636026 (+ 1776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:37.636053 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:37.637648 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:37.637654 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:37.637849 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:37.637885 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 12:07:37.638005 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 12:07:37.638140 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9009,"mutex_wait_us":54,"thread_start_us":9761,"threads_started":1}
I20260811 12:07:37.638576 5746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:37.621699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49160 (local address 127.9.221.130:40071)
0811 12:07:37.631984 (+ 10285us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:37.631994 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:37.632023 (+ 29us) server_negotiation.cc:407] Connection header received
0811 12:07:37.632130 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:37.632137 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:37.632357 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:37.632554 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:07:37.632920 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:37.634098 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:37.636234 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:37.637482 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:37.637639 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:37.637861 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:37.638053 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 12:07:37.638208 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10101,"thread_start_us":10326,"threads_started":1}
I20260811 12:07:37.639866 5620 tablet_service.cc:1467] Processing CreateTablet for tablet b0e4a86d768b408185f29f9a8ed55f44 (DEFAULT_TABLE table=test-table [id=99735be780c54fb1b91090122fe1bb57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:07:37.641419 5620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0e4a86d768b408185f29f9a8ed55f44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:07:37.641436 5422 tablet_service.cc:1467] Processing CreateTablet for tablet b0e4a86d768b408185f29f9a8ed55f44 (DEFAULT_TABLE table=test-table [id=99735be780c54fb1b91090122fe1bb57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:07:37.642710 5422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0e4a86d768b408185f29f9a8ed55f44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:07:37.648692 5318 heartbeater.cc:503] Master 127.9.221.190:33157 was elected leader, sending a full tablet report...
I20260811 12:07:37.653921 5750 tablet_bootstrap.cc:492] T b0e4a86d768b408185f29f9a8ed55f44 P 7658f50444e74655a24292c521371a08: Bootstrap starting.
I20260811 12:07:37.655216 5751 tablet_bootstrap.cc:492] T b0e4a86d768b408185f29f9a8ed55f44 P 9e4bec4edae943fba5fab62ae4851de3: Bootstrap starting.
I20260811 12:07:37.656742 5752 tablet_bootstrap.cc:492] T b0e4a86d768b408185f29f9a8ed55f44 P 24df44299c104ccd8c09590a86f86001: Bootstrap starting.
I20260811 12:07:37.658854 5751 tablet_bootstrap.cc:654] T b0e4a86d768b408185f29f9a8ed55f44 P 9e4bec4edae943fba5fab62ae4851de3: Neither blocks nor log segments found. Creating new log.
I20260811 12:07:37.658826 5750 tablet_bootstrap.cc:654] T b0e4a86d768b408185f29f9a8ed55f44 P 7658f50444e74655a24292c521371a08: Neither blocks nor log segments found. Creating new log.
I20260811 12:07:37.660183 5752 tablet_bootstrap.cc:654] T b0e4a86d768b408185f29f9a8ed55f44 P 24df44299c104ccd8c09590a86f86001: Neither blocks nor log segments found. Creating new log.
I20260811 12:07:37.660189 5751 log.cc:824] T b0e4a86d768b408185f29f9a8ed55f44 P 9e4bec4edae943fba5fab62ae4851de3: Log is configured to *not* fsync() on all Append() calls
I20260811 12:07:37.660194 5750 log.cc:824] T b0e4a86d768b408185f29f9a8ed55f44 P 7658f50444e74655a24292c521371a08: Log is configured to *not* fsync() on all Append() calls
I20260811 12:07:37.661420 5752 log.cc:824] T b0e4a86d768b408185f29f9a8ed55f44 P 24df44299c104ccd8c09590a86f86001: Log is configured to *not* fsync() on all Append() calls
I20260811 12:07:37.663503 5750 tablet_bootstrap.cc:492] T b0e4a86d768b408185f29f9a8ed55f44 P 7658f50444e74655a24292c521371a08: No bootstrap required, opened a new log
I20260811 12:07:37.663538 5751 tablet_bootstrap.cc:492] T b0e4a86d768b408185f29f9a8ed55f44 P 9e4bec4edae943fba5fab62ae4851de3: No bootstrap required, opened a new log
I20260811 12:07:37.663872 5751 ts_tablet_manager.cc:1397] T b0e4a86d768b408185f29f9a8ed55f44 P 9e4bec4edae943fba5fab62ae4851de3: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260811 12:07:37.663888 5750 ts_tablet_manager.cc:1397] T b0e4a86d768b408185f29f9a8ed55f44 P 7658f50444e74655a24292c521371a08: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260811 12:07:37.664829 5752 tablet_bootstrap.cc:492] T b0e4a86d768b408185f29f9a8ed55f44 P 24df44299c104ccd8c09590a86f86001: No bootstrap required, opened a new log
I20260811 12:07:37.665119 5752 ts_tablet_manager.cc:1397] T b0e4a86d768b408185f29f9a8ed55f44 P 24df44299c104ccd8c09590a86f86001: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.001s
I20260811 12:07:37.671265 5751 raft_consensus.cc:348] T b0e4a86d768b408185f29f9a8ed55f44 P 9e4bec4edae943fba5fab62ae4851de3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4bec4edae943fba5fab62ae4851de3" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43965 } } peers { permanent_uuid: "24df44299c104ccd8c09590a86f86001" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 35415 } } peers { permanent_uuid: "7658f50444e74655a24292c521371a08" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 40071 } }
I20260811 12:07:37.671654 5751 raft_consensus.cc:374] T b0e4a86d768b408185f29f9a8ed55f44 P 9e4bec4edae943fba5fab62ae4851de3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:07:37.671819 5751 raft_consensus.cc:729] T b0e4a86d768b408185f29f9a8ed55f44 P 9e4bec4edae943fba5fab62ae4851de3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e4bec4edae943fba5fab62ae4851de3, State: Initialized, Role: FOLLOWER
I20260811 12:07:37.672448 5751 consensus_queue.cc:260] T b0e4a86d768b408185f29f9a8ed55f44 P 9e4bec4edae943fba5fab62ae4851de3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4bec4edae943fba5fab62ae4851de3" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43965 } } peers { permanent_uuid: "24df44299c104ccd8c09590a86f86001" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 35415 } } peers { permanent_uuid: "7658f50444e74655a24292c521371a08" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 40071 } }
I20260811 12:07:37.675566 5751 ts_tablet_manager.cc:1428] T b0e4a86d768b408185f29f9a8ed55f44 P 9e4bec4edae943fba5fab62ae4851de3: Time spent starting tablet: real 0.012s user 0.010s sys 0.001s
I20260811 12:07:37.675385 5750 raft_consensus.cc:348] T b0e4a86d768b408185f29f9a8ed55f44 P 7658f50444e74655a24292c521371a08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4bec4edae943fba5fab62ae4851de3" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43965 } } peers { permanent_uuid: "24df44299c104ccd8c09590a86f86001" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 35415 } } peers { permanent_uuid: "7658f50444e74655a24292c521371a08" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 40071 } }
I20260811 12:07:37.675968 5750 raft_consensus.cc:374] T b0e4a86d768b408185f29f9a8ed55f44 P 7658f50444e74655a24292c521371a08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:07:37.676095 5750 raft_consensus.cc:729] T b0e4a86d768b408185f29f9a8ed55f44 P 7658f50444e74655a24292c521371a08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7658f50444e74655a24292c521371a08, State: Initialized, Role: FOLLOWER
I20260811 12:07:37.676512 5752 raft_consensus.cc:348] T b0e4a86d768b408185f29f9a8ed55f44 P 24df44299c104ccd8c09590a86f86001 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4bec4edae943fba5fab62ae4851de3" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43965 } } peers { permanent_uuid: "24df44299c104ccd8c09590a86f86001" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 35415 } } peers { permanent_uuid: "7658f50444e74655a24292c521371a08" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 40071 } }
I20260811 12:07:37.677002 5752 raft_consensus.cc:374] T b0e4a86d768b408185f29f9a8ed55f44 P 24df44299c104ccd8c09590a86f86001 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:07:37.676791 5750 consensus_queue.cc:260] T b0e4a86d768b408185f29f9a8ed55f44 P 7658f50444e74655a24292c521371a08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4bec4edae943fba5fab62ae4851de3" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43965 } } peers { permanent_uuid: "24df44299c104ccd8c09590a86f86001" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 35415 } } peers { permanent_uuid: "7658f50444e74655a24292c521371a08" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 40071 } }
I20260811 12:07:37.677109 5752 raft_consensus.cc:729] T b0e4a86d768b408185f29f9a8ed55f44 P 24df44299c104ccd8c09590a86f86001 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24df44299c104ccd8c09590a86f86001, State: Initialized, Role: FOLLOWER
I20260811 12:07:37.678028 5522 heartbeater.cc:503] Master 127.9.221.190:33157 was elected leader, sending a full tablet report...
I20260811 12:07:37.677896 5752 consensus_queue.cc:260] T b0e4a86d768b408185f29f9a8ed55f44 P 24df44299c104ccd8c09590a86f86001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4bec4edae943fba5fab62ae4851de3" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43965 } } peers { permanent_uuid: "24df44299c104ccd8c09590a86f86001" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 35415 } } peers { permanent_uuid: "7658f50444e74655a24292c521371a08" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 40071 } }
I20260811 12:07:37.679415 5718 heartbeater.cc:503] Master 127.9.221.190:33157 was elected leader, sending a full tablet report...
I20260811 12:07:37.680552 5750 ts_tablet_manager.cc:1428] T b0e4a86d768b408185f29f9a8ed55f44 P 7658f50444e74655a24292c521371a08: Time spent starting tablet: real 0.017s user 0.015s sys 0.000s
I20260811 12:07:37.680644 5752 ts_tablet_manager.cc:1428] T b0e4a86d768b408185f29f9a8ed55f44 P 24df44299c104ccd8c09590a86f86001: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
I20260811 12:07:37.798509 5762 raft_consensus.cc:482] T b0e4a86d768b408185f29f9a8ed55f44 P 24df44299c104ccd8c09590a86f86001 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:07:37.798857 5762 raft_consensus.cc:504] T b0e4a86d768b408185f29f9a8ed55f44 P 24df44299c104ccd8c09590a86f86001 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4bec4edae943fba5fab62ae4851de3" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43965 } } peers { permanent_uuid: "24df44299c104ccd8c09590a86f86001" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 35415 } } peers { permanent_uuid: "7658f50444e74655a24292c521371a08" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 40071 } }
I20260811 12:07:37.800786 5762 leader_election.cc:288] T b0e4a86d768b408185f29f9a8ed55f44 P 24df44299c104ccd8c09590a86f86001 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e4bec4edae943fba5fab62ae4851de3 (127.9.221.129:43965), 7658f50444e74655a24292c521371a08 (127.9.221.130:40071)
W20260811 12:07:37.807788 5719 tablet.cc:2369] T b0e4a86d768b408185f29f9a8ed55f44 P 24df44299c104ccd8c09590a86f86001: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:07:37.808449 5720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:37.801219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:43965 (local address 127.9.221.131:36645)
0811 12:07:37.801568 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:37.801590 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:07:37.801750 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:07:37.802229 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:07:37.802234 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:07:37.802255 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:07:37.802452 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:37.802460 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:37.803717 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:37.803720 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:37.805930 (+ 2210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:37.805944 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:37.807512 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:37.807520 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:37.807732 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:37.807783 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 12:07:37.807928 (+ 145us) client_negotiation.cc:240] Negotiation successful
0811 12:07:37.808074 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"mutex_wait_us":3}
I20260811 12:07:37.808631 5746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:37.801470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:44109 (local address 127.9.221.130:40071)
0811 12:07:37.801729 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:37.801736 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:37.802223 (+ 487us) server_negotiation.cc:407] Connection header received
0811 12:07:37.802399 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:37.802403 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:37.802474 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:37.802594 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:07:37.803083 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:37.803988 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:37.806438 (+ 2450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:37.807722 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:37.807878 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:37.807942 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:37.808279 (+ 337us) server_negotiation.cc:299] Negotiation successful
0811 12:07:37.808399 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 12:07:37.808981 5741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:37.801178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:36645 (local address 127.9.221.129:43965)
0811 12:07:37.801395 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:37.801402 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:37.801907 (+ 505us) server_negotiation.cc:407] Connection header received
0811 12:07:37.801998 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:37.802003 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:37.802079 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:37.802149 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:07:37.802600 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:37.803561 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:37.806071 (+ 2510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:37.807312 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:37.807485 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:37.807559 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:37.808143 (+ 584us) server_negotiation.cc:299] Negotiation successful
0811 12:07:37.808289 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 12:07:37.809296 5763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:37.801343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.130:40071 (local address 127.9.221.131:44109)
0811 12:07:37.802070 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:37.802085 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:07:37.802236 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:07:37.802623 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:07:37.802626 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:07:37.802645 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:07:37.802933 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:37.802940 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:37.804140 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:37.804145 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:37.806187 (+ 2042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:37.806197 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:37.807905 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:37.807913 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:37.808094 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:37.808127 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 12:07:37.808805 (+ 678us) client_negotiation.cc:240] Negotiation successful
0811 12:07:37.808947 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"mutex_wait_us":49,"thread_start_us":407,"threads_started":1}
I20260811 12:07:37.809904 5250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0e4a86d768b408185f29f9a8ed55f44" candidate_uuid: "24df44299c104ccd8c09590a86f86001" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e4bec4edae943fba5fab62ae4851de3" is_pre_election: true
I20260811 12:07:37.810389 5453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0e4a86d768b408185f29f9a8ed55f44" candidate_uuid: "24df44299c104ccd8c09590a86f86001" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7658f50444e74655a24292c521371a08" is_pre_election: true
I20260811 12:07:37.810545 5250 raft_consensus.cc:2454] T b0e4a86d768b408185f29f9a8ed55f44 P 9e4bec4edae943fba5fab62ae4851de3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24df44299c104ccd8c09590a86f86001 in term 0.
I20260811 12:07:37.810928 5453 raft_consensus.cc:2454] T b0e4a86d768b408185f29f9a8ed55f44 P 7658f50444e74655a24292c521371a08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24df44299c104ccd8c09590a86f86001 in term 0.
I20260811 12:07:37.811326 5551 leader_election.cc:302] T b0e4a86d768b408185f29f9a8ed55f44 P 24df44299c104ccd8c09590a86f86001 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24df44299c104ccd8c09590a86f86001, 9e4bec4edae943fba5fab62ae4851de3; no voters:
I20260811 12:07:37.811810 5762 raft_consensus.cc:2781] T b0e4a86d768b408185f29f9a8ed55f44 P 24df44299c104ccd8c09590a86f86001 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:07:37.811965 5762 raft_consensus.cc:482] T b0e4a86d768b408185f29f9a8ed55f44 P 24df44299c104ccd8c09590a86f86001 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:07:37.812090 5762 raft_consensus.cc:3037] T b0e4a86d768b408185f29f9a8ed55f44 P 24df44299c104ccd8c09590a86f86001 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:07:37.814348 5762 raft_consensus.cc:504] T b0e4a86d768b408185f29f9a8ed55f44 P 24df44299c104ccd8c09590a86f86001 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4bec4edae943fba5fab62ae4851de3" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43965 } } peers { permanent_uuid: "24df44299c104ccd8c09590a86f86001" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 35415 } } peers { permanent_uuid: "7658f50444e74655a24292c521371a08" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 40071 } }
I20260811 12:07:37.814968 5762 leader_election.cc:288] T b0e4a86d768b408185f29f9a8ed55f44 P 24df44299c104ccd8c09590a86f86001 [CANDIDATE]: Term 1 election: Requested vote from peers 9e4bec4edae943fba5fab62ae4851de3 (127.9.221.129:43965), 7658f50444e74655a24292c521371a08 (127.9.221.130:40071)
I20260811 12:07:37.815405 5453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0e4a86d768b408185f29f9a8ed55f44" candidate_uuid: "24df44299c104ccd8c09590a86f86001" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7658f50444e74655a24292c521371a08"
I20260811 12:07:37.815415 5250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0e4a86d768b408185f29f9a8ed55f44" candidate_uuid: "24df44299c104ccd8c09590a86f86001" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e4bec4edae943fba5fab62ae4851de3"
I20260811 12:07:37.815593 5453 raft_consensus.cc:3037] T b0e4a86d768b408185f29f9a8ed55f44 P 7658f50444e74655a24292c521371a08 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:07:37.815613 5250 raft_consensus.cc:3037] T b0e4a86d768b408185f29f9a8ed55f44 P 9e4bec4edae943fba5fab62ae4851de3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:07:37.817868 5250 raft_consensus.cc:2454] T b0e4a86d768b408185f29f9a8ed55f44 P 9e4bec4edae943fba5fab62ae4851de3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24df44299c104ccd8c09590a86f86001 in term 1.
I20260811 12:07:37.817898 5453 raft_consensus.cc:2454] T b0e4a86d768b408185f29f9a8ed55f44 P 7658f50444e74655a24292c521371a08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24df44299c104ccd8c09590a86f86001 in term 1.
I20260811 12:07:37.818282 5556 leader_election.cc:302] T b0e4a86d768b408185f29f9a8ed55f44 P 24df44299c104ccd8c09590a86f86001 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24df44299c104ccd8c09590a86f86001, 7658f50444e74655a24292c521371a08; no voters:
I20260811 12:07:37.818571 5762 raft_consensus.cc:2781] T b0e4a86d768b408185f29f9a8ed55f44 P 24df44299c104ccd8c09590a86f86001 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:07:37.819026 5762 raft_consensus.cc:686] T b0e4a86d768b408185f29f9a8ed55f44 P 24df44299c104ccd8c09590a86f86001 [term 1 LEADER]: Becoming Leader. State: Replica: 24df44299c104ccd8c09590a86f86001, State: Running, Role: LEADER
I20260811 12:07:37.819502 5762 consensus_queue.cc:237] T b0e4a86d768b408185f29f9a8ed55f44 P 24df44299c104ccd8c09590a86f86001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4bec4edae943fba5fab62ae4851de3" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43965 } } peers { permanent_uuid: "24df44299c104ccd8c09590a86f86001" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 35415 } } peers { permanent_uuid: "7658f50444e74655a24292c521371a08" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 40071 } }
I20260811 12:07:37.824304 5022 catalog_manager.cc:5495] T b0e4a86d768b408185f29f9a8ed55f44 P 24df44299c104ccd8c09590a86f86001 reported cstate change: term changed from 0 to 1, leader changed from <none> to 24df44299c104ccd8c09590a86f86001 (127.9.221.131). New cstate: current_term: 1 leader_uuid: "24df44299c104ccd8c09590a86f86001" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4bec4edae943fba5fab62ae4851de3" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24df44299c104ccd8c09590a86f86001" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 35415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7658f50444e74655a24292c521371a08" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 40071 } health_report { overall_health: UNKNOWN } } }
W20260811 12:07:37.835470 5523 tablet.cc:2369] T b0e4a86d768b408185f29f9a8ed55f44 P 7658f50444e74655a24292c521371a08: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:07:37.868304 5743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:37.862751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47934 (local address 127.9.221.131:35415)
0811 12:07:37.862935 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:37.862940 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:37.862957 (+ 17us) server_negotiation.cc:407] Connection header received
0811 12:07:37.863086 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:37.863089 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:37.863193 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:37.863334 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:07:37.863725 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:37.864702 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:37.866045 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:37.866447 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:37.866615 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:37.867129 (+ 514us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:07:37.867608 (+ 479us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:07:37.867834 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:37.867932 (+ 98us) server_negotiation.cc:299] Negotiation successful
0811 12:07:37.868009 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260811 12:07:37.876827 5319 tablet.cc:2369] T b0e4a86d768b408185f29f9a8ed55f44 P 9e4bec4edae943fba5fab62ae4851de3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:07:37.878409 5453 raft_consensus.cc:1261] T b0e4a86d768b408185f29f9a8ed55f44 P 7658f50444e74655a24292c521371a08 [term 1 FOLLOWER]: Refusing update from remote peer 24df44299c104ccd8c09590a86f86001: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:07:37.878458 5250 raft_consensus.cc:1261] T b0e4a86d768b408185f29f9a8ed55f44 P 9e4bec4edae943fba5fab62ae4851de3 [term 1 FOLLOWER]: Refusing update from remote peer 24df44299c104ccd8c09590a86f86001: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:07:37.879459 5764 consensus_queue.cc:1035] T b0e4a86d768b408185f29f9a8ed55f44 P 24df44299c104ccd8c09590a86f86001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e4bec4edae943fba5fab62ae4851de3" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:07:37.879993 5762 consensus_queue.cc:1035] T b0e4a86d768b408185f29f9a8ed55f44 P 24df44299c104ccd8c09590a86f86001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7658f50444e74655a24292c521371a08" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 40071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:07:37.891933 5767 mvcc.cc:205] Tried to move back new op lower bound from 7317299437059248128 to 7317299436832477184. Current Snapshot: MvccSnapshot[applied={T|T < 7317299437059248128}]
I20260811 12:07:37.894308 5770 mvcc.cc:205] Tried to move back new op lower bound from 7317299437059248128 to 7317299436832477184. Current Snapshot: MvccSnapshot[applied={T|T < 7317299437059248128}]
I20260811 12:07:37.908591 5022 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39992:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:07:37.908900 5022 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39992:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:07:37.912897 5022 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5715fbad9dd04b709d7fae42b1be8200: Sending DeleteTablet for 3 replicas of tablet b0e4a86d768b408185f29f9a8ed55f44
I20260811 12:07:37.914049 5217 tablet_service.cc:1514] Processing DeleteTablet for tablet b0e4a86d768b408185f29f9a8ed55f44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:07:37 UTC) from {username='slave'} at 127.0.0.1:48476
I20260811 12:07:37.914153 5620 tablet_service.cc:1514] Processing DeleteTablet for tablet b0e4a86d768b408185f29f9a8ed55f44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:07:37 UTC) from {username='slave'} at 127.0.0.1:47922
I20260811 12:07:37.914180 5422 tablet_service.cc:1514] Processing DeleteTablet for tablet b0e4a86d768b408185f29f9a8ed55f44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:07:37 UTC) from {username='slave'} at 127.0.0.1:49160
I20260811 12:07:37.920044 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 5112
I20260811 12:07:37.927408 5786 tablet_replica.cc:330] T b0e4a86d768b408185f29f9a8ed55f44 P 7658f50444e74655a24292c521371a08: stopping tablet replica
I20260811 12:07:37.927951 5786 raft_consensus.cc:2229] T b0e4a86d768b408185f29f9a8ed55f44 P 7658f50444e74655a24292c521371a08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:07:37.928544 5786 raft_consensus.cc:2258] T b0e4a86d768b408185f29f9a8ed55f44 P 7658f50444e74655a24292c521371a08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:07:37.929551 5784 tablet_replica.cc:330] T b0e4a86d768b408185f29f9a8ed55f44 P 9e4bec4edae943fba5fab62ae4851de3: stopping tablet replica
I20260811 12:07:37.929937 5784 raft_consensus.cc:2229] T b0e4a86d768b408185f29f9a8ed55f44 P 9e4bec4edae943fba5fab62ae4851de3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:07:37.930545 5784 raft_consensus.cc:2258] T b0e4a86d768b408185f29f9a8ed55f44 P 9e4bec4edae943fba5fab62ae4851de3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:07:37.931095 5786 ts_tablet_manager.cc:1905] T b0e4a86d768b408185f29f9a8ed55f44 P 7658f50444e74655a24292c521371a08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:07:37.933099 5784 ts_tablet_manager.cc:1905] T b0e4a86d768b408185f29f9a8ed55f44 P 9e4bec4edae943fba5fab62ae4851de3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:07:37.936338 5787 tablet_replica.cc:330] T b0e4a86d768b408185f29f9a8ed55f44 P 24df44299c104ccd8c09590a86f86001: stopping tablet replica
I20260811 12:07:37.936743 5787 raft_consensus.cc:2229] T b0e4a86d768b408185f29f9a8ed55f44 P 24df44299c104ccd8c09590a86f86001 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:07:37.937400 5787 raft_consensus.cc:2258] T b0e4a86d768b408185f29f9a8ed55f44 P 24df44299c104ccd8c09590a86f86001 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:07:37.939674 5787 ts_tablet_manager.cc:1905] T b0e4a86d768b408185f29f9a8ed55f44 P 24df44299c104ccd8c09590a86f86001: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:07:37.941176 5784 ts_tablet_manager.cc:1918] T b0e4a86d768b408185f29f9a8ed55f44 P 9e4bec4edae943fba5fab62ae4851de3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:07:37.941282 5786 ts_tablet_manager.cc:1918] T b0e4a86d768b408185f29f9a8ed55f44 P 7658f50444e74655a24292c521371a08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:07:37.941461 5786 log.cc:1196] T b0e4a86d768b408185f29f9a8ed55f44 P 7658f50444e74655a24292c521371a08: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-1/wal/wals/b0e4a86d768b408185f29f9a8ed55f44
I20260811 12:07:37.941461 5784 log.cc:1196] T b0e4a86d768b408185f29f9a8ed55f44 P 9e4bec4edae943fba5fab62ae4851de3: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-0/wal/wals/b0e4a86d768b408185f29f9a8ed55f44
I20260811 12:07:37.942075 5786 ts_tablet_manager.cc:1939] T b0e4a86d768b408185f29f9a8ed55f44 P 7658f50444e74655a24292c521371a08: Deleting consensus metadata
I20260811 12:07:37.942226 5784 ts_tablet_manager.cc:1939] T b0e4a86d768b408185f29f9a8ed55f44 P 9e4bec4edae943fba5fab62ae4851de3: Deleting consensus metadata
I20260811 12:07:37.944141 4994 catalog_manager.cc:4841] TS 9e4bec4edae943fba5fab62ae4851de3 (127.9.221.129:43965): tablet b0e4a86d768b408185f29f9a8ed55f44 (table test-table [id=99735be780c54fb1b91090122fe1bb57]) successfully deleted
I20260811 12:07:37.946408 5787 ts_tablet_manager.cc:1918] T b0e4a86d768b408185f29f9a8ed55f44 P 24df44299c104ccd8c09590a86f86001: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:07:37.946609 5787 log.cc:1196] T b0e4a86d768b408185f29f9a8ed55f44 P 24df44299c104ccd8c09590a86f86001: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449743084027-10102-0/minicluster-data/ts-2/wal/wals/b0e4a86d768b408185f29f9a8ed55f44
I20260811 12:07:37.947101 5787 ts_tablet_manager.cc:1939] T b0e4a86d768b408185f29f9a8ed55f44 P 24df44299c104ccd8c09590a86f86001: Deleting consensus metadata
I20260811 12:07:37.948468 5000 catalog_manager.cc:4841] TS 7658f50444e74655a24292c521371a08 (127.9.221.130:40071): tablet b0e4a86d768b408185f29f9a8ed55f44 (table test-table [id=99735be780c54fb1b91090122fe1bb57]) successfully deleted
I20260811 12:07:37.948755 4995 catalog_manager.cc:4841] TS 24df44299c104ccd8c09590a86f86001 (127.9.221.131:35415): tablet b0e4a86d768b408185f29f9a8ed55f44 (table test-table [id=99735be780c54fb1b91090122fe1bb57]) successfully deleted
I20260811 12:07:37.951712 5741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:37.921316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48478 (local address 127.9.221.129:43965)
0811 12:07:37.921515 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:37.921522 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:37.931316 (+ 9794us) server_negotiation.cc:407] Connection header received
0811 12:07:37.931563 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:37.931569 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:37.931713 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:37.931844 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:37.944388 (+ 12544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:37.945738 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:37.948012 (+ 2274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:37.948626 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:37.948760 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:37.950303 (+ 1543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:37.950312 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:37.950326 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:07:37.950904 (+ 578us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:37.950949 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:07:37.950968 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:37.951054 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:37.951249 (+ 195us) server_negotiation.cc:299] Negotiation successful
0811 12:07:37.951337 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 12:07:37.952348 5312 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:07:38.058367 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 5112
I20260811 12:07:38.085973 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 5327
I20260811 12:07:38.095420 5746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:38.087086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49174 (local address 127.9.221.130:40071)
0811 12:07:38.087285 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:38.087291 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:38.087324 (+ 33us) server_negotiation.cc:407] Connection header received
0811 12:07:38.087397 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:38.087402 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:38.087509 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:38.087608 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:38.088188 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:38.089388 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:38.091503 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:38.092198 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:38.092325 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:38.093640 (+ 1315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:38.093658 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:38.093677 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:07:38.094405 (+ 728us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:38.094475 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:07:38.094502 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:38.094605 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:38.094782 (+ 177us) server_negotiation.cc:299] Negotiation successful
0811 12:07:38.094889 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 12:07:38.096333 5512 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:07:38.186415 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 5327
I20260811 12:07:38.220916 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 5525
I20260811 12:07:38.229190 5743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:38.221833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47946 (local address 127.9.221.131:35415)
0811 12:07:38.222017 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:38.222021 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:38.222134 (+ 113us) server_negotiation.cc:407] Connection header received
0811 12:07:38.222358 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:38.222362 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:38.222454 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:38.222551 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:38.223054 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:38.223916 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:38.225957 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:38.226366 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:38.226453 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:38.227801 (+ 1348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:38.227809 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:38.227822 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:07:38.228363 (+ 541us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:38.228407 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:07:38.228431 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:38.228509 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:38.228719 (+ 210us) server_negotiation.cc:299] Negotiation successful
0811 12:07:38.228801 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 12:07:38.229934 5710 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:07:38.349642 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 5525
I20260811 12:07:38.377388 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 4972
I20260811 12:07:38.378599 5067 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:07:38.487454 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 4972
2026-08-11T12:07:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2992 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:07:38 dist-test-slave-mjjr krb5kdc[5798](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:07:38 dist-test-slave-mjjr krb5kdc[5798](info): set up 2 sockets
Aug 11 12:07:38 dist-test-slave-mjjr krb5kdc[5798](info): commencing operation
krb5kdc: starting...
W20260811 12:07:39.834990 10102 mini_kdc.cc:121] Time spent starting KDC: real 1.296s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Aug 11 12:07:40 dist-test-slave-mjjr krb5kdc[5798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:07:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:07:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.221.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
I20260811 12:07:40.154594 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:41673
--webserver_interface=127.9.221.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:36009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:41673
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:07:40.502249 5817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:07:40.502731 5817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:07:40.502910 5817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:07:40.510684 5817 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:07:40.510816 5817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:07:40.510865 5817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:07:40.510923 5817 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:07:40.510968 5817 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:07:40.521308 5817 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:36009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:41673
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:41673
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.221.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:07:40.522689 5817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:07:40.524165 5817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:07:40.536270 5822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:40.536167 5823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:40.536201 5825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:07:40.537259 5817 server_base.cc:1034] running on GCE node
I20260811 12:07:40.538020 5817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:07:40.539517 5817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:07:40.540756 5817 hybrid_clock.cc:648] HybridClock initialized: now 1786450060540726 us; error 66 us; skew 500 ppm
I20260811 12:07:40.541129 5817 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:07:40 dist-test-slave-mjjr krb5kdc[5798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450060, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:07:40.549062 5817 init.cc:377] Logged in from keytab as kudu/127.9.221.190@KRBTEST.COM (short username kudu)
I20260811 12:07:40.550796 5817 webserver.cc:460] Webserver started at http://127.9.221.190:38277/ using document root <none> and password file <none>
I20260811 12:07:40.551549 5817 fs_manager.cc:359] Metadata directory not provided
I20260811 12:07:40.551755 5817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:07:40.552114 5817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:07:40.555447 5817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "1e2d2af677de425eafff93dc5b3c12d0"
format_stamp: "Formatted at 2026-08-11 12:07:40 on dist-test-slave-mjjr"
server_key: "7c9c2c7962d0a64c6300cf84f093a914"
server_key_iv: "bc59f0e9b1b15e9658d2bf5d81c3a3e9"
server_key_version: "encryptionkey@0"
I20260811 12:07:40.556354 5817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "1e2d2af677de425eafff93dc5b3c12d0"
format_stamp: "Formatted at 2026-08-11 12:07:40 on dist-test-slave-mjjr"
server_key: "7c9c2c7962d0a64c6300cf84f093a914"
server_key_iv: "bc59f0e9b1b15e9658d2bf5d81c3a3e9"
server_key_version: "encryptionkey@0"
I20260811 12:07:40.561964 5817 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 12:07:40.565709 5837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:40.567101 5817 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:07:40.567322 5817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "1e2d2af677de425eafff93dc5b3c12d0"
format_stamp: "Formatted at 2026-08-11 12:07:40 on dist-test-slave-mjjr"
server_key: "7c9c2c7962d0a64c6300cf84f093a914"
server_key_iv: "bc59f0e9b1b15e9658d2bf5d81c3a3e9"
server_key_version: "encryptionkey@0"
I20260811 12:07:40.567538 5817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:07:40.586004 5817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:07:40.590447 5817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:07:40.590747 5817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:07:40.610352 5817 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.190:41673
I20260811 12:07:40.610389 5924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.190:41673 every 8 connection(s)
I20260811 12:07:40.611789 5817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:07:40.615389 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 5817
I20260811 12:07:40.615532 5927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:07:40.615581 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:07:40.616066 10102 external_mini_cluster.cc:1428] Setting key 56b6065348fa8c66492ae5aedab9833e
I20260811 12:07:40.630007 5927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e2d2af677de425eafff93dc5b3c12d0: Bootstrap starting.
Aug 11 12:07:40 dist-test-slave-mjjr krb5kdc[5798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:07:40.633410 5927 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e2d2af677de425eafff93dc5b3c12d0: Neither blocks nor log segments found. Creating new log.
I20260811 12:07:40.634614 5927 log.cc:824] T 00000000000000000000000000000000 P 1e2d2af677de425eafff93dc5b3c12d0: Log is configured to *not* fsync() on all Append() calls
I20260811 12:07:40.637514 5927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e2d2af677de425eafff93dc5b3c12d0: No bootstrap required, opened a new log
I20260811 12:07:40.639449 5936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:40.617942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36090 (local address 127.9.221.190:41673)
0811 12:07:40.618839 (+ 897us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:40.618850 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:40.618890 (+ 40us) server_negotiation.cc:407] Connection header received
0811 12:07:40.620437 (+ 1547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:40.620458 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:40.620824 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:40.621131 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:40.623059 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:40.624424 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:40.626390 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:40.627094 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:40.627238 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:40.632253 (+ 5015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:40.632260 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:40.632273 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:07:40.632308 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:40.635473 (+ 3165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:40.636189 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:40.636196 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:40.636202 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:40.636371 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:40.636764 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:40.636770 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:40.636773 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:40.637252 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:40.637603 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:40.638038 (+ 435us) server_negotiation.cc:299] Negotiation successful
0811 12:07:40.638440 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":407,"threads_started":1}
I20260811 12:07:40.645702 5927 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e2d2af677de425eafff93dc5b3c12d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e2d2af677de425eafff93dc5b3c12d0" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 41673 } }
I20260811 12:07:40.646063 5927 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e2d2af677de425eafff93dc5b3c12d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:07:40.646183 5927 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e2d2af677de425eafff93dc5b3c12d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e2d2af677de425eafff93dc5b3c12d0, State: Initialized, Role: FOLLOWER
I20260811 12:07:40.646733 5927 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e2d2af677de425eafff93dc5b3c12d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e2d2af677de425eafff93dc5b3c12d0" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 41673 } }
I20260811 12:07:40.646983 5927 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e2d2af677de425eafff93dc5b3c12d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:07:40.647127 5927 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e2d2af677de425eafff93dc5b3c12d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:07:40.647300 5927 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e2d2af677de425eafff93dc5b3c12d0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:07:40.649466 5927 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e2d2af677de425eafff93dc5b3c12d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e2d2af677de425eafff93dc5b3c12d0" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 41673 } }
I20260811 12:07:40.650031 5927 leader_election.cc:302] T 00000000000000000000000000000000 P 1e2d2af677de425eafff93dc5b3c12d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e2d2af677de425eafff93dc5b3c12d0; no voters:
I20260811 12:07:40.650586 5927 leader_election.cc:288] T 00000000000000000000000000000000 P 1e2d2af677de425eafff93dc5b3c12d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:07:40.650712 5938 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e2d2af677de425eafff93dc5b3c12d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:07:40.651929 5938 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e2d2af677de425eafff93dc5b3c12d0 [term 1 LEADER]: Becoming Leader. State: Replica: 1e2d2af677de425eafff93dc5b3c12d0, State: Running, Role: LEADER
I20260811 12:07:40.652484 5938 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e2d2af677de425eafff93dc5b3c12d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e2d2af677de425eafff93dc5b3c12d0" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 41673 } }
I20260811 12:07:40.652684 5927 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e2d2af677de425eafff93dc5b3c12d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:07:40.655390 5940 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e2d2af677de425eafff93dc5b3c12d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e2d2af677de425eafff93dc5b3c12d0. Latest consensus state: current_term: 1 leader_uuid: "1e2d2af677de425eafff93dc5b3c12d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e2d2af677de425eafff93dc5b3c12d0" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 41673 } } }
I20260811 12:07:40.655889 5940 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e2d2af677de425eafff93dc5b3c12d0 [sys.catalog]: This master's current role is: LEADER
I20260811 12:07:40.656199 5939 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e2d2af677de425eafff93dc5b3c12d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e2d2af677de425eafff93dc5b3c12d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e2d2af677de425eafff93dc5b3c12d0" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 41673 } } }
I20260811 12:07:40.656500 5939 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e2d2af677de425eafff93dc5b3c12d0 [sys.catalog]: This master's current role is: LEADER
I20260811 12:07:40.657960 5952 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:07:40.663195 5952 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:07:40.672123 5952 catalog_manager.cc:1347] Generated new cluster ID: ce74e0626d8d428fa5c5f71123bafef7
I20260811 12:07:40.672300 5952 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:07:40.694235 5952 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:07:40.696017 5952 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:07:40.702953 5952 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e2d2af677de425eafff93dc5b3c12d0: Generated new TSK 0
I20260811 12:07:40.703799 5952 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.221.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab.
Entry for principal kudu/127.9.221.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab.
I20260811 12:07:40.788970 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.129:0
--local_ip_for_outbound_sockets=127.9.221.129
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:41673
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab
--principal=kudu/127.9.221.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:36009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:07:41.136833 5970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:07:41.137125 5970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:07:41.137290 5970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:07:41.144912 5970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:07:41.145032 5970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:07:41.145153 5970 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.129
I20260811 12:07:41.155268 5970 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:36009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab
--principal=kudu/127.9.221.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:41673
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.129
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:07:41.156641 5970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:07:41.158218 5970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:07:41.171734 5978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:41.171762 5977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:41.172260 5984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:07:41.172519 5970 server_base.cc:1034] running on GCE node
I20260811 12:07:41.173142 5970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:07:41.174090 5970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:07:41.175294 5970 hybrid_clock.cc:648] HybridClock initialized: now 1786450061175238 us; error 55 us; skew 500 ppm
I20260811 12:07:41.175611 5970 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:07:41 dist-test-slave-mjjr krb5kdc[5798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450061, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:07:41.178795 5970 init.cc:377] Logged in from keytab as kudu/127.9.221.129@KRBTEST.COM (short username kudu)
I20260811 12:07:41.180222 5970 webserver.cc:460] Webserver started at http://127.9.221.129:35937/ using document root <none> and password file <none>
I20260811 12:07:41.180868 5970 fs_manager.cc:359] Metadata directory not provided
I20260811 12:07:41.180969 5970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:07:41.181229 5970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:07:41.184136 5970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-0/data/instance:
uuid: "a1db98c733ce4dc58855ede36240ea75"
format_stamp: "Formatted at 2026-08-11 12:07:41 on dist-test-slave-mjjr"
server_key: "2bfb7ea51583e37b0d599e5fc2a42642"
server_key_iv: "d588642f92ca1260b24835bf7705b043"
server_key_version: "encryptionkey@0"
I20260811 12:07:41.184834 5970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance:
uuid: "a1db98c733ce4dc58855ede36240ea75"
format_stamp: "Formatted at 2026-08-11 12:07:41 on dist-test-slave-mjjr"
server_key: "2bfb7ea51583e37b0d599e5fc2a42642"
server_key_iv: "d588642f92ca1260b24835bf7705b043"
server_key_version: "encryptionkey@0"
I20260811 12:07:41.189997 5970 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 12:07:41.193429 5997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:41.194721 5970 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 12:07:41.194898 5970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-0/wal
uuid: "a1db98c733ce4dc58855ede36240ea75"
format_stamp: "Formatted at 2026-08-11 12:07:41 on dist-test-slave-mjjr"
server_key: "2bfb7ea51583e37b0d599e5fc2a42642"
server_key_iv: "d588642f92ca1260b24835bf7705b043"
server_key_version: "encryptionkey@0"
I20260811 12:07:41.195091 5970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:07:41.206615 5970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:07:41.210429 5970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:07:41.210712 5970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:07:41.211687 5970 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:07:41.212955 5970 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:07:41.213059 5970 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:41.213204 5970 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:07:41.213281 5970 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:41.246050 5970 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.129:33883
I20260811 12:07:41.246104 6168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.129:33883 every 8 connection(s)
I20260811 12:07:41.247377 5970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
I20260811 12:07:41.249965 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 5970
I20260811 12:07:41.250192 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance
I20260811 12:07:41.250697 10102 external_mini_cluster.cc:1428] Setting key 01d1548f3fa9c9512773b475e88e0c68
Aug 11 12:07:41 dist-test-slave-mjjr krb5kdc[5798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450061, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:07:41.269371 6173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:41.250003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:33167 (local address 127.9.221.190:41673)
0811 12:07:41.250506 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:41.250512 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:41.251666 (+ 1154us) server_negotiation.cc:407] Connection header received
0811 12:07:41.253861 (+ 2195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:41.253870 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:41.253962 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:41.254106 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:41.255747 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:41.256931 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:41.258464 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:41.259163 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:41.259306 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:41.262697 (+ 3391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:41.262705 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:41.262713 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:07:41.262759 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:41.266094 (+ 3335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:41.267052 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:41.267056 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:41.267059 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:41.267169 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:41.267619 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:41.267626 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:41.267634 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:41.267952 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:41.268112 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:41.268782 (+ 670us) server_negotiation.cc:299] Negotiation successful
0811 12:07:41.269002 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":238,"threads_started":1}
I20260811 12:07:41.270254 6174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:41.250638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:41673 (local address 127.9.221.129:33167)
0811 12:07:41.251464 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:41.251521 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 12:07:41.253558 (+ 2037us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:07:41.254384 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:07:41.254391 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:07:41.254932 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:07:41.255551 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:41.255570 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:41.257067 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:41.257070 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:41.258317 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:41.258329 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:41.259364 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:41.259368 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:41.259571 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:41.260309 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:07:41.260334 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:07:41.262487 (+ 2153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:07:41.266241 (+ 3754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:07:41.266252 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:07:41.266271 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:07:41.266943 (+ 672us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:07:41.267292 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:07:41.267298 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:07:41.267302 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:07:41.267494 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:07:41.268085 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:07:41.268093 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:07:41.268413 (+ 320us) client_negotiation.cc:769] Sending connection context
0811 12:07:41.268664 (+ 251us) client_negotiation.cc:240] Negotiation successful
0811 12:07:41.269171 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":361,"threads_started":1}
I20260811 12:07:41.271849 6170 heartbeater.cc:347] Connected to a master server at 127.9.221.190:41673
I20260811 12:07:41.272162 6170 heartbeater.cc:464] Registering TS with master...
I20260811 12:07:41.272832 6170 heartbeater.cc:511] Master 127.9.221.190:41673 requested a full tablet report, sending...
I20260811 12:07:41.274843 5870 ts_manager.cc:194] Registered new tserver with Master: a1db98c733ce4dc58855ede36240ea75 (127.9.221.129:33883)
I20260811 12:07:41.276348 5870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.129@KRBTEST.COM'} at 127.9.221.129:33167
WARNING: no policy specified for kudu/127.9.221.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab.
Entry for principal kudu/127.9.221.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab.
I20260811 12:07:41.318666 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.130:0
--local_ip_for_outbound_sockets=127.9.221.130
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:41673
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab
--principal=kudu/127.9.221.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:36009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:07:41.669883 6178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:07:41.670157 6178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:07:41.670253 6178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:07:41.677361 6178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:07:41.677474 6178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:07:41.677659 6178 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.130
I20260811 12:07:41.687095 6178 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:36009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab
--principal=kudu/127.9.221.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:41673
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.130
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:07:41.688370 6178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:07:41.689621 6178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:07:41.703104 6194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:41.703075 6195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:41.703167 6198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:07:41.704470 6178 server_base.cc:1034] running on GCE node
I20260811 12:07:41.705058 6178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:07:41.706005 6178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:07:41.707232 6178 hybrid_clock.cc:648] HybridClock initialized: now 1786450061707183 us; error 46 us; skew 500 ppm
I20260811 12:07:41.707541 6178 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:07:41 dist-test-slave-mjjr krb5kdc[5798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450061, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:07:41.710642 6178 init.cc:377] Logged in from keytab as kudu/127.9.221.130@KRBTEST.COM (short username kudu)
I20260811 12:07:41.712047 6178 webserver.cc:460] Webserver started at http://127.9.221.130:37487/ using document root <none> and password file <none>
I20260811 12:07:41.712685 6178 fs_manager.cc:359] Metadata directory not provided
I20260811 12:07:41.712831 6178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:07:41.713150 6178 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:07:41.715961 6178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-1/data/instance:
uuid: "e5705d321f264ec7833cf16bf878b16f"
format_stamp: "Formatted at 2026-08-11 12:07:41 on dist-test-slave-mjjr"
server_key: "1e3486fac5853afbaafb338c527c9c0c"
server_key_iv: "e7580f7f56b93980277d35af7551a652"
server_key_version: "encryptionkey@0"
I20260811 12:07:41.716704 6178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance:
uuid: "e5705d321f264ec7833cf16bf878b16f"
format_stamp: "Formatted at 2026-08-11 12:07:41 on dist-test-slave-mjjr"
server_key: "1e3486fac5853afbaafb338c527c9c0c"
server_key_iv: "e7580f7f56b93980277d35af7551a652"
server_key_version: "encryptionkey@0"
I20260811 12:07:41.721613 6178 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 12:07:41.724982 6209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:41.726226 6178 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:07:41.726411 6178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-1/wal
uuid: "e5705d321f264ec7833cf16bf878b16f"
format_stamp: "Formatted at 2026-08-11 12:07:41 on dist-test-slave-mjjr"
server_key: "1e3486fac5853afbaafb338c527c9c0c"
server_key_iv: "e7580f7f56b93980277d35af7551a652"
server_key_version: "encryptionkey@0"
I20260811 12:07:41.726629 6178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:07:41.741001 6178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:07:41.745051 6178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:07:41.745344 6178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:07:41.746318 6178 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:07:41.747735 6178 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:07:41.747844 6178 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:41.748004 6178 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:07:41.748078 6178 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:41.782197 6178 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.130:41719
I20260811 12:07:41.782287 6379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.130:41719 every 8 connection(s)
I20260811 12:07:41.783504 6178 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
I20260811 12:07:41.789778 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 6178
I20260811 12:07:41.789978 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance
I20260811 12:07:41.790426 10102 external_mini_cluster.cc:1428] Setting key 341eacd0efaf10d180d119a67856b626
Aug 11 12:07:41 dist-test-slave-mjjr krb5kdc[5798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450061, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.130@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:07:41.804168 6386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:41.786195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:38591 (local address 127.9.221.190:41673)
0811 12:07:41.786668 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:41.786672 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:41.787620 (+ 948us) server_negotiation.cc:407] Connection header received
0811 12:07:41.789497 (+ 1877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:41.789503 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:41.789588 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:41.789708 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:41.791276 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:41.792138 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:41.793625 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:41.794037 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:41.794170 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:41.797781 (+ 3611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:41.797786 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:41.797789 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:07:41.798031 (+ 242us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:41.801059 (+ 3028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:41.801810 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:41.801816 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:41.801820 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:41.801956 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:41.802398 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:41.802403 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:41.802407 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:41.802673 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:41.802831 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:41.803512 (+ 681us) server_negotiation.cc:299] Negotiation successful
0811 12:07:41.803779 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":246,"threads_started":1}
I20260811 12:07:41.805151 6387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:41.786770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:41673 (local address 127.9.221.130:38591)
0811 12:07:41.787450 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:41.787487 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 12:07:41.789218 (+ 1731us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:07:41.789980 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:07:41.789996 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:07:41.790557 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:07:41.791100 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:41.791116 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:41.792283 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:41.792288 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:41.793479 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:41.793491 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:41.794268 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:41.794275 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:41.794493 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:41.795159 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:07:41.795189 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:07:41.797555 (+ 2366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:07:41.801209 (+ 3654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:07:41.801218 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:07:41.801233 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:07:41.801672 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:07:41.802078 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:07:41.802083 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:07:41.802087 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:07:41.802280 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:07:41.802811 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:07:41.802818 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:07:41.803161 (+ 343us) client_negotiation.cc:769] Sending connection context
0811 12:07:41.803429 (+ 268us) client_negotiation.cc:240] Negotiation successful
0811 12:07:41.803999 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":261,"threads_started":1}
I20260811 12:07:41.806900 6382 heartbeater.cc:347] Connected to a master server at 127.9.221.190:41673
I20260811 12:07:41.807221 6382 heartbeater.cc:464] Registering TS with master...
I20260811 12:07:41.808029 6382 heartbeater.cc:511] Master 127.9.221.190:41673 requested a full tablet report, sending...
I20260811 12:07:41.809597 5870 ts_manager.cc:194] Registered new tserver with Master: e5705d321f264ec7833cf16bf878b16f (127.9.221.130:41719)
I20260811 12:07:41.810895 5870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.130@KRBTEST.COM'} at 127.9.221.130:38591
WARNING: no policy specified for kudu/127.9.221.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab.
Entry for principal kudu/127.9.221.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab.
I20260811 12:07:41.858412 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.131:0
--local_ip_for_outbound_sockets=127.9.221.131
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:41673
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab
--principal=kudu/127.9.221.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:36009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:07:42.205444 6390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:07:42.205706 6390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:07:42.205866 6390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:07:42.213238 6390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:07:42.213349 6390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:07:42.213569 6390 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.131
I20260811 12:07:42.223448 6390 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:36009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab
--principal=kudu/127.9.221.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:41673
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.131
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:07:42.224790 6390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:07:42.226119 6390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:07:42.242455 6396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:42.242779 6400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:42.242385 6398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:07:42.243949 6390 server_base.cc:1034] running on GCE node
I20260811 12:07:42.244980 6390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:07:42.246321 6390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:07:42.247630 6390 hybrid_clock.cc:648] HybridClock initialized: now 1786450062247560 us; error 51 us; skew 500 ppm
I20260811 12:07:42.248087 6390 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:07:42 dist-test-slave-mjjr krb5kdc[5798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450062, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:07:42.252496 6390 init.cc:377] Logged in from keytab as kudu/127.9.221.131@KRBTEST.COM (short username kudu)
I20260811 12:07:42.254257 6390 webserver.cc:460] Webserver started at http://127.9.221.131:43729/ using document root <none> and password file <none>
I20260811 12:07:42.254978 6390 fs_manager.cc:359] Metadata directory not provided
I20260811 12:07:42.255113 6390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:07:42.255447 6390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:07:42.258544 6390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-2/data/instance:
uuid: "ff07c861a7904edd9e776a2fd96a7f9a"
format_stamp: "Formatted at 2026-08-11 12:07:42 on dist-test-slave-mjjr"
server_key: "8e33cb1b6b04ced565ef013120f1b308"
server_key_iv: "d5da11e83621730d705d7b1fd39c1967"
server_key_version: "encryptionkey@0"
I20260811 12:07:42.259397 6390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance:
uuid: "ff07c861a7904edd9e776a2fd96a7f9a"
format_stamp: "Formatted at 2026-08-11 12:07:42 on dist-test-slave-mjjr"
server_key: "8e33cb1b6b04ced565ef013120f1b308"
server_key_iv: "d5da11e83621730d705d7b1fd39c1967"
server_key_version: "encryptionkey@0"
I20260811 12:07:42.266247 6390 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260811 12:07:42.270169 6408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:42.271818 6390 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:07:42.271994 6390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-2/wal
uuid: "ff07c861a7904edd9e776a2fd96a7f9a"
format_stamp: "Formatted at 2026-08-11 12:07:42 on dist-test-slave-mjjr"
server_key: "8e33cb1b6b04ced565ef013120f1b308"
server_key_iv: "d5da11e83621730d705d7b1fd39c1967"
server_key_version: "encryptionkey@0"
I20260811 12:07:42.272312 6390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:07:42.279248 6170 heartbeater.cc:503] Master 127.9.221.190:41673 was elected leader, sending a full tablet report...
I20260811 12:07:42.293105 6390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:07:42.297605 6390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:07:42.297849 6390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:07:42.298990 6390 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:07:42.300544 6390 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:07:42.300619 6390 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:42.300772 6390 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:07:42.300822 6390 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:42.346567 6390 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.131:37837
I20260811 12:07:42.346740 6592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.131:37837 every 8 connection(s)
I20260811 12:07:42.348037 6390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
I20260811 12:07:42.351197 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 6390
I20260811 12:07:42.351426 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance
I20260811 12:07:42.352114 10102 external_mini_cluster.cc:1428] Setting key a419e131412ee4ff4fc52b1b0adb9922
Aug 11 12:07:42 dist-test-slave-mjjr krb5kdc[5798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450062, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.131@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:07:42.373993 6599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:42.351030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:60293 (local address 127.9.221.190:41673)
0811 12:07:42.359692 (+ 8662us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:42.359701 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:42.359718 (+ 17us) server_negotiation.cc:407] Connection header received
0811 12:07:42.359781 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:42.359784 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:42.359853 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:42.359935 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:42.361448 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:42.362223 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:42.363784 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:42.364183 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:42.364275 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:42.368264 (+ 3989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:42.368268 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:42.368271 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:07:42.368300 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:42.370813 (+ 2513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:42.371558 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:42.371562 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:42.371564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:42.371763 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:42.372220 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:42.372225 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:42.372228 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:42.372515 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:42.372681 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:42.373384 (+ 703us) server_negotiation.cc:299] Negotiation successful
0811 12:07:42.373605 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8508,"thread_start_us":8386,"threads_started":1}
I20260811 12:07:42.374918 6600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:42.351777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:41673 (local address 127.9.221.131:60293)
0811 12:07:42.356667 (+ 4890us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:42.356798 (+ 131us) client_negotiation.cc:174] Beginning negotiation
0811 12:07:42.359108 (+ 2310us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:07:42.360170 (+ 1062us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:07:42.360178 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:07:42.360727 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:07:42.361267 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:42.361283 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:42.362374 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:42.362377 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:42.363585 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:42.363597 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:42.364389 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:42.364394 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:42.364595 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:42.365300 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:07:42.365334 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:07:42.367947 (+ 2613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:07:42.370950 (+ 3003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:07:42.370956 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:07:42.370968 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:07:42.371471 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:07:42.371881 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:07:42.371886 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:07:42.371890 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:07:42.372095 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:07:42.372646 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:07:42.372653 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:07:42.373005 (+ 352us) client_negotiation.cc:769] Sending connection context
0811 12:07:42.373282 (+ 277us) client_negotiation.cc:240] Negotiation successful
0811 12:07:42.373801 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4575,"thread_start_us":4314,"threads_started":1}
I20260811 12:07:42.376433 6594 heartbeater.cc:347] Connected to a master server at 127.9.221.190:41673
I20260811 12:07:42.376761 6594 heartbeater.cc:464] Registering TS with master...
I20260811 12:07:42.377612 6594 heartbeater.cc:511] Master 127.9.221.190:41673 requested a full tablet report, sending...
I20260811 12:07:42.379172 5870 ts_manager.cc:194] Registered new tserver with Master: ff07c861a7904edd9e776a2fd96a7f9a (127.9.221.131:37837)
I20260811 12:07:42.380237 5870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.131@KRBTEST.COM'} at 127.9.221.131:60293
I20260811 12:07:42.381000 10102 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:07:42.398505 6599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:42.385858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36096 (local address 127.9.221.190:41673)
0811 12:07:42.386038 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:42.386042 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:42.386498 (+ 456us) server_negotiation.cc:407] Connection header received
0811 12:07:42.386652 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:42.386656 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:42.386735 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:42.386824 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:42.388295 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:42.388975 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:42.390265 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:42.390663 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:42.390758 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:42.393480 (+ 2722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:42.393485 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:42.393488 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:07:42.393517 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:42.395961 (+ 2444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:42.396655 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:42.396658 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:42.396660 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:42.396768 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:42.397169 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:42.397173 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:42.397175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:42.397510 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:42.397778 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:42.398083 (+ 305us) server_negotiation.cc:299] Negotiation successful
0811 12:07:42.398228 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 12:07:42.402393 5870 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36096:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:07:42.409283 5870 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:07:42.437935 6614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:42.421363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57456 (local address 127.9.221.130:41719)
0811 12:07:42.422188 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:42.422201 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:42.423190 (+ 989us) server_negotiation.cc:407] Connection header received
0811 12:07:42.423459 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:42.423479 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:42.423887 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:42.425066 (+ 1179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:07:42.426479 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:42.428693 (+ 2214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:42.432877 (+ 4184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:42.434933 (+ 2056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:42.435158 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:42.435489 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:42.437111 (+ 1622us) server_negotiation.cc:299] Negotiation successful
0811 12:07:42.437413 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":620,"thread_start_us":492,"threads_started":1}
I20260811 12:07:42.438489 6613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:42.421317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.130:41719 (local address 127.0.0.1:57456)
0811 12:07:42.422739 (+ 1422us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:42.422780 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 12:07:42.422963 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:07:42.425437 (+ 2474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:07:42.425447 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:07:42.425486 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:07:42.426133 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:42.426157 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:42.428889 (+ 2732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:42.428895 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:42.432620 (+ 3725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:42.432641 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:42.435747 (+ 3106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:42.435760 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:42.436135 (+ 375us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:42.436238 (+ 103us) client_negotiation.cc:769] Sending connection context
0811 12:07:42.436438 (+ 200us) client_negotiation.cc:240] Negotiation successful
0811 12:07:42.436755 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1134,"thread_start_us":923,"threads_started":1}
I20260811 12:07:42.445644 6276 tablet_service.cc:1467] Processing CreateTablet for tablet 45959fbc4bc04fd2abc46ae66a88667a (DEFAULT_TABLE table=test-table [id=e7f7d43e77f744cbae479f86c2775fb1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:07:42.447717 6276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45959fbc4bc04fd2abc46ae66a88667a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:07:42.453106 6617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:42.423833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:33883 (local address 127.0.0.1:60434)
0811 12:07:42.440716 (+ 16883us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:42.440751 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 12:07:42.440929 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:07:42.442355 (+ 1426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:07:42.442365 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:07:42.442410 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:07:42.442989 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:42.443008 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:42.445547 (+ 2539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:42.445557 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:42.449068 (+ 3511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:42.449095 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:42.452003 (+ 2908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:42.452012 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:42.452298 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:42.452346 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 12:07:42.452464 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 12:07:42.452619 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16685,"mutex_wait_us":1,"thread_start_us":16446,"threads_started":1}
I20260811 12:07:42.454123 6618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:42.424529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60434 (local address 127.9.221.129:33883)
0811 12:07:42.440043 (+ 15514us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:42.440058 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:42.441116 (+ 1058us) server_negotiation.cc:407] Connection header received
0811 12:07:42.441294 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:42.441309 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:42.441677 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:42.441968 (+ 291us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:07:42.443415 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:42.445295 (+ 1880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:42.449835 (+ 4540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:42.451310 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:42.451489 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:42.451808 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:42.453464 (+ 1656us) server_negotiation.cc:299] Negotiation successful
0811 12:07:42.453683 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15235,"thread_start_us":14957,"threads_started":1}
I20260811 12:07:42.458844 6613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:42.422411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.131:37837 (local address 127.0.0.1:50476)
0811 12:07:42.438774 (+ 16363us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:42.438798 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 12:07:42.438927 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:07:42.450993 (+ 12066us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:07:42.451002 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:07:42.451044 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:07:42.451291 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:42.451305 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:42.453739 (+ 2434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:42.453744 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:42.455634 (+ 1890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:42.455684 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:42.458036 (+ 2352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:42.458042 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:42.458240 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:42.458286 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 12:07:42.458393 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 12:07:42.458570 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16243,"thread_start_us":27092,"threads_started":1}
I20260811 12:07:42.458362 6071 tablet_service.cc:1467] Processing CreateTablet for tablet 45959fbc4bc04fd2abc46ae66a88667a (DEFAULT_TABLE table=test-table [id=e7f7d43e77f744cbae479f86c2775fb1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:07:42.459468 6616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:42.422760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50476 (local address 127.9.221.131:37837)
0811 12:07:42.450370 (+ 27610us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:42.450379 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:42.450413 (+ 34us) server_negotiation.cc:407] Connection header received
0811 12:07:42.450508 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:42.450515 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:42.450746 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:42.450963 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:07:42.451556 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:42.453575 (+ 2019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:42.455936 (+ 2361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:42.457828 (+ 1892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:42.457992 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:42.458260 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:42.458460 (+ 200us) server_negotiation.cc:299] Negotiation successful
0811 12:07:42.458653 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27443,"thread_start_us":27216,"threads_started":1}
I20260811 12:07:42.460134 6071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45959fbc4bc04fd2abc46ae66a88667a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:07:42.462998 6619 tablet_bootstrap.cc:492] T 45959fbc4bc04fd2abc46ae66a88667a P e5705d321f264ec7833cf16bf878b16f: Bootstrap starting.
I20260811 12:07:42.463666 6480 tablet_service.cc:1467] Processing CreateTablet for tablet 45959fbc4bc04fd2abc46ae66a88667a (DEFAULT_TABLE table=test-table [id=e7f7d43e77f744cbae479f86c2775fb1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:07:42.465350 6480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45959fbc4bc04fd2abc46ae66a88667a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:07:42.466243 6619 tablet_bootstrap.cc:654] T 45959fbc4bc04fd2abc46ae66a88667a P e5705d321f264ec7833cf16bf878b16f: Neither blocks nor log segments found. Creating new log.
I20260811 12:07:42.467264 6619 log.cc:824] T 45959fbc4bc04fd2abc46ae66a88667a P e5705d321f264ec7833cf16bf878b16f: Log is configured to *not* fsync() on all Append() calls
I20260811 12:07:42.469801 6619 tablet_bootstrap.cc:492] T 45959fbc4bc04fd2abc46ae66a88667a P e5705d321f264ec7833cf16bf878b16f: No bootstrap required, opened a new log
I20260811 12:07:42.470080 6619 ts_tablet_manager.cc:1397] T 45959fbc4bc04fd2abc46ae66a88667a P e5705d321f264ec7833cf16bf878b16f: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 12:07:42.477155 6621 tablet_bootstrap.cc:492] T 45959fbc4bc04fd2abc46ae66a88667a P a1db98c733ce4dc58855ede36240ea75: Bootstrap starting.
I20260811 12:07:42.479127 6619 raft_consensus.cc:348] T 45959fbc4bc04fd2abc46ae66a88667a P e5705d321f264ec7833cf16bf878b16f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5705d321f264ec7833cf16bf878b16f" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 41719 } } peers { permanent_uuid: "ff07c861a7904edd9e776a2fd96a7f9a" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 37837 } } peers { permanent_uuid: "a1db98c733ce4dc58855ede36240ea75" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33883 } }
I20260811 12:07:42.479570 6619 raft_consensus.cc:374] T 45959fbc4bc04fd2abc46ae66a88667a P e5705d321f264ec7833cf16bf878b16f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:07:42.479700 6619 raft_consensus.cc:729] T 45959fbc4bc04fd2abc46ae66a88667a P e5705d321f264ec7833cf16bf878b16f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5705d321f264ec7833cf16bf878b16f, State: Initialized, Role: FOLLOWER
I20260811 12:07:42.480325 6619 consensus_queue.cc:260] T 45959fbc4bc04fd2abc46ae66a88667a P e5705d321f264ec7833cf16bf878b16f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5705d321f264ec7833cf16bf878b16f" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 41719 } } peers { permanent_uuid: "ff07c861a7904edd9e776a2fd96a7f9a" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 37837 } } peers { permanent_uuid: "a1db98c733ce4dc58855ede36240ea75" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33883 } }
I20260811 12:07:42.480931 6621 tablet_bootstrap.cc:654] T 45959fbc4bc04fd2abc46ae66a88667a P a1db98c733ce4dc58855ede36240ea75: Neither blocks nor log segments found. Creating new log.
I20260811 12:07:42.481619 6622 tablet_bootstrap.cc:492] T 45959fbc4bc04fd2abc46ae66a88667a P ff07c861a7904edd9e776a2fd96a7f9a: Bootstrap starting.
I20260811 12:07:42.483237 6382 heartbeater.cc:503] Master 127.9.221.190:41673 was elected leader, sending a full tablet report...
I20260811 12:07:42.483641 6621 log.cc:824] T 45959fbc4bc04fd2abc46ae66a88667a P a1db98c733ce4dc58855ede36240ea75: Log is configured to *not* fsync() on all Append() calls
I20260811 12:07:42.484053 6619 ts_tablet_manager.cc:1428] T 45959fbc4bc04fd2abc46ae66a88667a P e5705d321f264ec7833cf16bf878b16f: Time spent starting tablet: real 0.014s user 0.009s sys 0.005s
I20260811 12:07:42.486953 6622 tablet_bootstrap.cc:654] T 45959fbc4bc04fd2abc46ae66a88667a P ff07c861a7904edd9e776a2fd96a7f9a: Neither blocks nor log segments found. Creating new log.
I20260811 12:07:42.487600 6621 tablet_bootstrap.cc:492] T 45959fbc4bc04fd2abc46ae66a88667a P a1db98c733ce4dc58855ede36240ea75: No bootstrap required, opened a new log
I20260811 12:07:42.487896 6621 ts_tablet_manager.cc:1397] T 45959fbc4bc04fd2abc46ae66a88667a P a1db98c733ce4dc58855ede36240ea75: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.005s
I20260811 12:07:42.488528 6622 log.cc:824] T 45959fbc4bc04fd2abc46ae66a88667a P ff07c861a7904edd9e776a2fd96a7f9a: Log is configured to *not* fsync() on all Append() calls
I20260811 12:07:42.495518 6621 raft_consensus.cc:348] T 45959fbc4bc04fd2abc46ae66a88667a P a1db98c733ce4dc58855ede36240ea75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5705d321f264ec7833cf16bf878b16f" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 41719 } } peers { permanent_uuid: "ff07c861a7904edd9e776a2fd96a7f9a" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 37837 } } peers { permanent_uuid: "a1db98c733ce4dc58855ede36240ea75" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33883 } }
I20260811 12:07:42.495929 6621 raft_consensus.cc:374] T 45959fbc4bc04fd2abc46ae66a88667a P a1db98c733ce4dc58855ede36240ea75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:07:42.496042 6621 raft_consensus.cc:729] T 45959fbc4bc04fd2abc46ae66a88667a P a1db98c733ce4dc58855ede36240ea75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1db98c733ce4dc58855ede36240ea75, State: Initialized, Role: FOLLOWER
I20260811 12:07:42.496737 6621 consensus_queue.cc:260] T 45959fbc4bc04fd2abc46ae66a88667a P a1db98c733ce4dc58855ede36240ea75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5705d321f264ec7833cf16bf878b16f" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 41719 } } peers { permanent_uuid: "ff07c861a7904edd9e776a2fd96a7f9a" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 37837 } } peers { permanent_uuid: "a1db98c733ce4dc58855ede36240ea75" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33883 } }
I20260811 12:07:42.498612 6621 ts_tablet_manager.cc:1428] T 45959fbc4bc04fd2abc46ae66a88667a P a1db98c733ce4dc58855ede36240ea75: Time spent starting tablet: real 0.011s user 0.008s sys 0.002s
I20260811 12:07:42.499251 6622 tablet_bootstrap.cc:492] T 45959fbc4bc04fd2abc46ae66a88667a P ff07c861a7904edd9e776a2fd96a7f9a: No bootstrap required, opened a new log
I20260811 12:07:42.499627 6622 ts_tablet_manager.cc:1397] T 45959fbc4bc04fd2abc46ae66a88667a P ff07c861a7904edd9e776a2fd96a7f9a: Time spent bootstrapping tablet: real 0.018s user 0.010s sys 0.005s
W20260811 12:07:42.499826 6171 tablet.cc:2369] T 45959fbc4bc04fd2abc46ae66a88667a P a1db98c733ce4dc58855ede36240ea75: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:07:42.509979 6622 raft_consensus.cc:348] T 45959fbc4bc04fd2abc46ae66a88667a P ff07c861a7904edd9e776a2fd96a7f9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5705d321f264ec7833cf16bf878b16f" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 41719 } } peers { permanent_uuid: "ff07c861a7904edd9e776a2fd96a7f9a" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 37837 } } peers { permanent_uuid: "a1db98c733ce4dc58855ede36240ea75" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33883 } }
I20260811 12:07:42.510463 6622 raft_consensus.cc:374] T 45959fbc4bc04fd2abc46ae66a88667a P ff07c861a7904edd9e776a2fd96a7f9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:07:42.510597 6622 raft_consensus.cc:729] T 45959fbc4bc04fd2abc46ae66a88667a P ff07c861a7904edd9e776a2fd96a7f9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff07c861a7904edd9e776a2fd96a7f9a, State: Initialized, Role: FOLLOWER
I20260811 12:07:42.511252 6622 consensus_queue.cc:260] T 45959fbc4bc04fd2abc46ae66a88667a P ff07c861a7904edd9e776a2fd96a7f9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5705d321f264ec7833cf16bf878b16f" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 41719 } } peers { permanent_uuid: "ff07c861a7904edd9e776a2fd96a7f9a" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 37837 } } peers { permanent_uuid: "a1db98c733ce4dc58855ede36240ea75" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33883 } }
I20260811 12:07:42.512358 6594 heartbeater.cc:503] Master 127.9.221.190:41673 was elected leader, sending a full tablet report...
I20260811 12:07:42.513002 6622 ts_tablet_manager.cc:1428] T 45959fbc4bc04fd2abc46ae66a88667a P ff07c861a7904edd9e776a2fd96a7f9a: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
W20260811 12:07:42.535692 6385 tablet.cc:2369] T 45959fbc4bc04fd2abc46ae66a88667a P e5705d321f264ec7833cf16bf878b16f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:07:42.555696 6633 raft_consensus.cc:482] T 45959fbc4bc04fd2abc46ae66a88667a P ff07c861a7904edd9e776a2fd96a7f9a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:07:42.556028 6633 raft_consensus.cc:504] T 45959fbc4bc04fd2abc46ae66a88667a P ff07c861a7904edd9e776a2fd96a7f9a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5705d321f264ec7833cf16bf878b16f" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 41719 } } peers { permanent_uuid: "ff07c861a7904edd9e776a2fd96a7f9a" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 37837 } } peers { permanent_uuid: "a1db98c733ce4dc58855ede36240ea75" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33883 } }
I20260811 12:07:42.557859 6633 leader_election.cc:288] T 45959fbc4bc04fd2abc46ae66a88667a P ff07c861a7904edd9e776a2fd96a7f9a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5705d321f264ec7833cf16bf878b16f (127.9.221.130:41719), a1db98c733ce4dc58855ede36240ea75 (127.9.221.129:33883)
I20260811 12:07:42.564616 6600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:42.558157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.130:41719 (local address 127.9.221.131:56641)
0811 12:07:42.558355 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:42.558373 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:07:42.558515 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:07:42.559109 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:07:42.559115 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:07:42.559155 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:07:42.559358 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:42.559369 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:42.560946 (+ 1577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:42.560950 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:42.562787 (+ 1837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:42.562802 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:42.564085 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:42.564089 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:42.564223 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:42.564247 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 12:07:42.564322 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 12:07:42.564422 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 12:07:42.564785 6614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:42.558420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:56641 (local address 127.9.221.130:41719)
0811 12:07:42.558663 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:42.558667 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:42.558683 (+ 16us) server_negotiation.cc:407] Connection header received
0811 12:07:42.558742 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:42.558745 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:42.558830 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:42.558950 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:07:42.559576 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:42.560802 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:42.562942 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:42.563944 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:42.564095 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:42.564170 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:42.564380 (+ 210us) server_negotiation.cc:299] Negotiation successful
0811 12:07:42.564521 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 12:07:42.565613 6618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:42.558780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:55501 (local address 127.9.221.129:33883)
0811 12:07:42.559010 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:42.559016 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:42.559442 (+ 426us) server_negotiation.cc:407] Connection header received
0811 12:07:42.559600 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:42.559607 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:42.559724 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:42.559818 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:07:42.560561 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:42.561381 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:42.563502 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:42.564709 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:42.564866 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:42.564947 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:42.565356 (+ 409us) server_negotiation.cc:299] Negotiation successful
0811 12:07:42.565451 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 12:07:42.565775 6634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:42.558619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:33883 (local address 127.9.221.131:55501)
0811 12:07:42.559306 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:42.559325 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:07:42.559469 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:07:42.559861 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:07:42.559868 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:07:42.559933 (+ 65us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:07:42.560361 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:42.560376 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:42.561533 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:42.561538 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:42.563351 (+ 1813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:42.563361 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:42.565021 (+ 1660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:42.565027 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:42.565163 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:42.565185 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 12:07:42.565300 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 12:07:42.565445 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":324,"threads_started":1}
I20260811 12:07:42.565775 6311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45959fbc4bc04fd2abc46ae66a88667a" candidate_uuid: "ff07c861a7904edd9e776a2fd96a7f9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5705d321f264ec7833cf16bf878b16f" is_pre_election: true
I20260811 12:07:42.566398 6311 raft_consensus.cc:2454] T 45959fbc4bc04fd2abc46ae66a88667a P e5705d321f264ec7833cf16bf878b16f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff07c861a7904edd9e776a2fd96a7f9a in term 0.
I20260811 12:07:42.566735 6100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45959fbc4bc04fd2abc46ae66a88667a" candidate_uuid: "ff07c861a7904edd9e776a2fd96a7f9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1db98c733ce4dc58855ede36240ea75" is_pre_election: true
I20260811 12:07:42.567142 6412 leader_election.cc:302] T 45959fbc4bc04fd2abc46ae66a88667a P ff07c861a7904edd9e776a2fd96a7f9a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e5705d321f264ec7833cf16bf878b16f, ff07c861a7904edd9e776a2fd96a7f9a; no voters:
I20260811 12:07:42.567286 6100 raft_consensus.cc:2454] T 45959fbc4bc04fd2abc46ae66a88667a P a1db98c733ce4dc58855ede36240ea75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff07c861a7904edd9e776a2fd96a7f9a in term 0.
I20260811 12:07:42.567612 6633 raft_consensus.cc:2781] T 45959fbc4bc04fd2abc46ae66a88667a P ff07c861a7904edd9e776a2fd96a7f9a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:07:42.567838 6633 raft_consensus.cc:482] T 45959fbc4bc04fd2abc46ae66a88667a P ff07c861a7904edd9e776a2fd96a7f9a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:07:42.567972 6633 raft_consensus.cc:3037] T 45959fbc4bc04fd2abc46ae66a88667a P ff07c861a7904edd9e776a2fd96a7f9a [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:07:42.570237 6633 raft_consensus.cc:504] T 45959fbc4bc04fd2abc46ae66a88667a P ff07c861a7904edd9e776a2fd96a7f9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5705d321f264ec7833cf16bf878b16f" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 41719 } } peers { permanent_uuid: "ff07c861a7904edd9e776a2fd96a7f9a" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 37837 } } peers { permanent_uuid: "a1db98c733ce4dc58855ede36240ea75" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33883 } }
I20260811 12:07:42.570837 6633 leader_election.cc:288] T 45959fbc4bc04fd2abc46ae66a88667a P ff07c861a7904edd9e776a2fd96a7f9a [CANDIDATE]: Term 1 election: Requested vote from peers e5705d321f264ec7833cf16bf878b16f (127.9.221.130:41719), a1db98c733ce4dc58855ede36240ea75 (127.9.221.129:33883)
I20260811 12:07:42.571321 6311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45959fbc4bc04fd2abc46ae66a88667a" candidate_uuid: "ff07c861a7904edd9e776a2fd96a7f9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5705d321f264ec7833cf16bf878b16f"
I20260811 12:07:42.571476 6100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45959fbc4bc04fd2abc46ae66a88667a" candidate_uuid: "ff07c861a7904edd9e776a2fd96a7f9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1db98c733ce4dc58855ede36240ea75"
I20260811 12:07:42.571547 6311 raft_consensus.cc:3037] T 45959fbc4bc04fd2abc46ae66a88667a P e5705d321f264ec7833cf16bf878b16f [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:07:42.571734 6100 raft_consensus.cc:3037] T 45959fbc4bc04fd2abc46ae66a88667a P a1db98c733ce4dc58855ede36240ea75 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:07:42.573992 6311 raft_consensus.cc:2454] T 45959fbc4bc04fd2abc46ae66a88667a P e5705d321f264ec7833cf16bf878b16f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff07c861a7904edd9e776a2fd96a7f9a in term 1.
I20260811 12:07:42.574261 6100 raft_consensus.cc:2454] T 45959fbc4bc04fd2abc46ae66a88667a P a1db98c733ce4dc58855ede36240ea75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff07c861a7904edd9e776a2fd96a7f9a in term 1.
I20260811 12:07:42.574489 6412 leader_election.cc:302] T 45959fbc4bc04fd2abc46ae66a88667a P ff07c861a7904edd9e776a2fd96a7f9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e5705d321f264ec7833cf16bf878b16f, ff07c861a7904edd9e776a2fd96a7f9a; no voters:
I20260811 12:07:42.574805 6633 raft_consensus.cc:2781] T 45959fbc4bc04fd2abc46ae66a88667a P ff07c861a7904edd9e776a2fd96a7f9a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:07:42.575392 6633 raft_consensus.cc:686] T 45959fbc4bc04fd2abc46ae66a88667a P ff07c861a7904edd9e776a2fd96a7f9a [term 1 LEADER]: Becoming Leader. State: Replica: ff07c861a7904edd9e776a2fd96a7f9a, State: Running, Role: LEADER
I20260811 12:07:42.576021 6633 consensus_queue.cc:237] T 45959fbc4bc04fd2abc46ae66a88667a P ff07c861a7904edd9e776a2fd96a7f9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5705d321f264ec7833cf16bf878b16f" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 41719 } } peers { permanent_uuid: "ff07c861a7904edd9e776a2fd96a7f9a" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 37837 } } peers { permanent_uuid: "a1db98c733ce4dc58855ede36240ea75" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33883 } }
I20260811 12:07:42.580647 5870 catalog_manager.cc:5495] T 45959fbc4bc04fd2abc46ae66a88667a P ff07c861a7904edd9e776a2fd96a7f9a reported cstate change: term changed from 0 to 1, leader changed from <none> to ff07c861a7904edd9e776a2fd96a7f9a (127.9.221.131). New cstate: current_term: 1 leader_uuid: "ff07c861a7904edd9e776a2fd96a7f9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5705d321f264ec7833cf16bf878b16f" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 41719 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff07c861a7904edd9e776a2fd96a7f9a" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 37837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a1db98c733ce4dc58855ede36240ea75" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33883 } health_report { overall_health: UNKNOWN } } }
W20260811 12:07:42.600236 6597 tablet.cc:2369] T 45959fbc4bc04fd2abc46ae66a88667a P ff07c861a7904edd9e776a2fd96a7f9a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:07:42.626521 6616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:42.619886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50488 (local address 127.9.221.131:37837)
0811 12:07:42.620073 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:42.620078 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:42.620096 (+ 18us) server_negotiation.cc:407] Connection header received
0811 12:07:42.620245 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:42.620253 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:42.620332 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:42.620460 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:07:42.620928 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:42.621633 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:42.623829 (+ 2196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:42.624419 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:42.624554 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:42.625282 (+ 728us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:07:42.625792 (+ 510us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:07:42.625953 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:42.626127 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 12:07:42.626214 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 12:07:42.635470 6100 raft_consensus.cc:1261] T 45959fbc4bc04fd2abc46ae66a88667a P a1db98c733ce4dc58855ede36240ea75 [term 1 FOLLOWER]: Refusing update from remote peer ff07c861a7904edd9e776a2fd96a7f9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:07:42.635473 6311 raft_consensus.cc:1261] T 45959fbc4bc04fd2abc46ae66a88667a P e5705d321f264ec7833cf16bf878b16f [term 1 FOLLOWER]: Refusing update from remote peer ff07c861a7904edd9e776a2fd96a7f9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:07:42.636593 6635 consensus_queue.cc:1035] T 45959fbc4bc04fd2abc46ae66a88667a P ff07c861a7904edd9e776a2fd96a7f9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1db98c733ce4dc58855ede36240ea75" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 33883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:07:42.637048 6633 consensus_queue.cc:1035] T 45959fbc4bc04fd2abc46ae66a88667a P ff07c861a7904edd9e776a2fd96a7f9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5705d321f264ec7833cf16bf878b16f" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 41719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:07:42.646430 6641 mvcc.cc:205] Tried to move back new op lower bound from 7317299456543834112 to 7317299456315899904. Current Snapshot: MvccSnapshot[applied={T|T < 7317299456543834112}]
I20260811 12:07:42.652019 6649 mvcc.cc:205] Tried to move back new op lower bound from 7317299456543834112 to 7317299456315899904. Current Snapshot: MvccSnapshot[applied={T|T < 7317299456543834112}]
I20260811 12:07:42.664768 5870 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:07:42.665064 5870 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:07:42.668697 5870 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1e2d2af677de425eafff93dc5b3c12d0: Sending DeleteTablet for 3 replicas of tablet 45959fbc4bc04fd2abc46ae66a88667a
I20260811 12:07:42.670013 6480 tablet_service.cc:1514] Processing DeleteTablet for tablet 45959fbc4bc04fd2abc46ae66a88667a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:07:42 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:50476
I20260811 12:07:42.670006 6071 tablet_service.cc:1514] Processing DeleteTablet for tablet 45959fbc4bc04fd2abc46ae66a88667a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:07:42 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:60434
I20260811 12:07:42.670113 6276 tablet_service.cc:1514] Processing DeleteTablet for tablet 45959fbc4bc04fd2abc46ae66a88667a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:07:42 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:57456
I20260811 12:07:42.675450 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 5970
I20260811 12:07:42.703393 6665 tablet_replica.cc:330] T 45959fbc4bc04fd2abc46ae66a88667a P e5705d321f264ec7833cf16bf878b16f: stopping tablet replica
I20260811 12:07:42.706563 6665 raft_consensus.cc:2229] T 45959fbc4bc04fd2abc46ae66a88667a P e5705d321f264ec7833cf16bf878b16f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:07:42.707309 6665 raft_consensus.cc:2258] T 45959fbc4bc04fd2abc46ae66a88667a P e5705d321f264ec7833cf16bf878b16f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:07:42.714102 6665 ts_tablet_manager.cc:1905] T 45959fbc4bc04fd2abc46ae66a88667a P e5705d321f264ec7833cf16bf878b16f: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 12:07:42 dist-test-slave-mjjr krb5kdc[5798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.129@KRBTEST.COM
I20260811 12:07:42.724249 6662 tablet_replica.cc:330] T 45959fbc4bc04fd2abc46ae66a88667a P a1db98c733ce4dc58855ede36240ea75: stopping tablet replica
I20260811 12:07:42.724653 6661 tablet_replica.cc:330] T 45959fbc4bc04fd2abc46ae66a88667a P ff07c861a7904edd9e776a2fd96a7f9a: stopping tablet replica
I20260811 12:07:42.724929 6662 raft_consensus.cc:2229] T 45959fbc4bc04fd2abc46ae66a88667a P a1db98c733ce4dc58855ede36240ea75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:07:42.725121 6661 raft_consensus.cc:2229] T 45959fbc4bc04fd2abc46ae66a88667a P ff07c861a7904edd9e776a2fd96a7f9a [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:07:42.725586 6662 raft_consensus.cc:2258] T 45959fbc4bc04fd2abc46ae66a88667a P a1db98c733ce4dc58855ede36240ea75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:07:42.725982 6661 raft_consensus.cc:2258] T 45959fbc4bc04fd2abc46ae66a88667a P ff07c861a7904edd9e776a2fd96a7f9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:07:42.728509 6661 ts_tablet_manager.cc:1905] T 45959fbc4bc04fd2abc46ae66a88667a P ff07c861a7904edd9e776a2fd96a7f9a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:07:42.730123 6665 ts_tablet_manager.cc:1918] T 45959fbc4bc04fd2abc46ae66a88667a P e5705d321f264ec7833cf16bf878b16f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:07:42.730372 6665 log.cc:1196] T 45959fbc4bc04fd2abc46ae66a88667a P e5705d321f264ec7833cf16bf878b16f: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-1/wal/wals/45959fbc4bc04fd2abc46ae66a88667a
I20260811 12:07:42.731138 6665 ts_tablet_manager.cc:1939] T 45959fbc4bc04fd2abc46ae66a88667a P e5705d321f264ec7833cf16bf878b16f: Deleting consensus metadata
I20260811 12:07:42.731123 6662 ts_tablet_manager.cc:1905] T 45959fbc4bc04fd2abc46ae66a88667a P a1db98c733ce4dc58855ede36240ea75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:07:42.733321 5844 catalog_manager.cc:4841] TS e5705d321f264ec7833cf16bf878b16f (127.9.221.130:41719): tablet 45959fbc4bc04fd2abc46ae66a88667a (table test-table [id=e7f7d43e77f744cbae479f86c2775fb1]) successfully deleted
I20260811 12:07:42.734688 6618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:42.677110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60436 (local address 127.9.221.129:33883)
0811 12:07:42.677342 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:42.677350 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:42.702588 (+ 25238us) server_negotiation.cc:407] Connection header received
0811 12:07:42.703936 (+ 1348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:42.703943 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:42.704051 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:42.704135 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:42.707874 (+ 3739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:42.709124 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:42.711488 (+ 2364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:42.712238 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:42.712341 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:42.725698 (+ 13357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:42.725716 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:42.725738 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:07:42.725794 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:42.729774 (+ 3980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:42.731808 (+ 2034us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:42.731817 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:42.731821 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:42.731997 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:42.732547 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:42.732553 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:42.732556 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:42.733033 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:42.733324 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:42.733999 (+ 675us) server_negotiation.cc:299] Negotiation successful
0811 12:07:42.734266 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 12:07:42.736269 6163 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:07:42.737709 6661 ts_tablet_manager.cc:1918] T 45959fbc4bc04fd2abc46ae66a88667a P ff07c861a7904edd9e776a2fd96a7f9a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:07:42.738044 6661 log.cc:1196] T 45959fbc4bc04fd2abc46ae66a88667a P ff07c861a7904edd9e776a2fd96a7f9a: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449743084027-10102-0/minicluster-data/ts-2/wal/wals/45959fbc4bc04fd2abc46ae66a88667a
I20260811 12:07:42.739099 6661 ts_tablet_manager.cc:1939] T 45959fbc4bc04fd2abc46ae66a88667a P ff07c861a7904edd9e776a2fd96a7f9a: Deleting consensus metadata
I20260811 12:07:42.741466 5838 catalog_manager.cc:4841] TS ff07c861a7904edd9e776a2fd96a7f9a (127.9.221.131:37837): tablet 45959fbc4bc04fd2abc46ae66a88667a (table test-table [id=e7f7d43e77f744cbae479f86c2775fb1]) successfully deleted
I20260811 12:07:42.853829 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 5970
W20260811 12:07:42.884476 5844 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.221.129:33883 (error 108)
I20260811 12:07:42.884644 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 6178
I20260811 12:07:42.886684 6613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:42.886147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:33883 (local address 127.0.0.1:60448)
0811 12:07:42.886342 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:42.886485 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.129:33883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260811 12:07:42.887024 5844 catalog_manager.cc:4584] TS a1db98c733ce4dc58855ede36240ea75 (127.9.221.129:33883): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 45959fbc4bc04fd2abc46ae66a88667a: Network error: Client connection negotiation failed: client connection to 127.9.221.129:33883: connect: Connection refused (error 111)
Aug 11 12:07:42 dist-test-slave-mjjr krb5kdc[5798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.130@KRBTEST.COM
I20260811 12:07:42.902974 6614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:42.886219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57468 (local address 127.9.221.130:41719)
0811 12:07:42.886403 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:42.886411 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:42.886432 (+ 21us) server_negotiation.cc:407] Connection header received
0811 12:07:42.886514 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:42.886520 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:42.886601 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:42.886732 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:42.888499 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:42.889246 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:42.891039 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:42.891412 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:42.891498 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:42.896851 (+ 5353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:42.896865 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:42.896881 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:07:42.896921 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:42.900147 (+ 3226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:42.900849 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:42.900857 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:42.900859 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:42.900987 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:42.901385 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:42.901394 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:42.901396 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:42.901774 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:42.902092 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:42.902422 (+ 330us) server_negotiation.cc:299] Negotiation successful
0811 12:07:42.902645 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 12:07:42.903617 6371 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:07:42.940762 6613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:42.940215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:33883 (local address 127.0.0.1:60450)
0811 12:07:42.940462 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:42.940579 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.129:33883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260811 12:07:42.942018 6613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:42.941569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:33883 (local address 127.0.0.1:60466)
0811 12:07:42.941744 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:42.941831 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.129:33883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 12:07:42.995811 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 6178
I20260811 12:07:43.002928 6613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:43.002408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:33883 (local address 127.0.0.1:60478)
0811 12:07:43.002624 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:43.002739 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.129:33883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 12:07:43.004457 6613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:43.004043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:33883 (local address 127.0.0.1:60488)
0811 12:07:43.004200 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:43.004290 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.129:33883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260811 12:07:43.022666 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 6390
Aug 11 12:07:43 dist-test-slave-mjjr krb5kdc[5798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.131@KRBTEST.COM
I20260811 12:07:43.039304 6616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:43.023562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50490 (local address 127.9.221.131:37837)
0811 12:07:43.023870 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:43.023876 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:43.023896 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:07:43.023970 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:43.023974 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:43.024046 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:43.024159 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:43.025498 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:43.026319 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:43.027672 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:43.028217 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:43.028398 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:43.033076 (+ 4678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:43.033091 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:43.033106 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:07:43.033139 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:43.036333 (+ 3194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:43.037062 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:43.037071 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:43.037076 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:43.037229 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:43.037623 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:43.037629 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:43.037632 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:43.038047 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:43.038403 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:43.038708 (+ 305us) server_negotiation.cc:299] Negotiation successful
0811 12:07:43.038943 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 12:07:43.040158 6582 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:07:43.086061 6613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:43.085500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:33883 (local address 127.0.0.1:60492)
0811 12:07:43.085752 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:43.085876 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.129:33883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260811 12:07:43.087356 6613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:43.086842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:33883 (local address 127.0.0.1:60506)
0811 12:07:43.087066 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:43.087152 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.129:33883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260811 12:07:43.168983 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 6390
I20260811 12:07:43.196651 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 5817
I20260811 12:07:43.197844 5920 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:07:43.309783 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 5817
I20260811 12:07:43.309870 6613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:43.309293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:33883 (local address 127.0.0.1:60510)
0811 12:07:43.309547 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:43.309685 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.129:33883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
2026-08-11T12:07:43Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4821 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-11T12:07:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:07:43Z Disabled control of system clock
I20260811 12:07:43.406122 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:44807
--webserver_interface=127.9.221.190
--webserver_port=0
--builtin_ntp_servers=127.9.221.148:35513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:44807
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 12:07:43.751386 6679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:07:43.751780 6679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:07:43.751945 6679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:07:43.759544 6679 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:07:43.759716 6679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:07:43.759804 6679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:07:43.759909 6679 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:07:43.759995 6679 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:07:43.769429 6679 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:35513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:44807
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:44807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.221.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:07:43.770814 6679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:07:43.772783 6679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:07:43.784492 6690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:43.784601 6691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:43.785013 6693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:07:43.785750 6679 server_base.cc:1034] running on GCE node
I20260811 12:07:43.786582 6679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:07:43.788017 6679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:07:43.789258 6679 hybrid_clock.cc:648] HybridClock initialized: now 1786450063789211 us; error 50 us; skew 500 ppm
I20260811 12:07:43.789602 6679 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:07:43.791842 6679 webserver.cc:460] Webserver started at http://127.9.221.190:41017/ using document root <none> and password file <none>
I20260811 12:07:43.792547 6679 fs_manager.cc:359] Metadata directory not provided
I20260811 12:07:43.792685 6679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:07:43.793020 6679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:07:43.795953 6679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "113251304764470b92d34432d9a97af4"
format_stamp: "Formatted at 2026-08-11 12:07:43 on dist-test-slave-mjjr"
server_key: "b9bb963c8ca92287097adef534f93ac1"
server_key_iv: "a2d73f07211346fe21fcf0412f77c561"
server_key_version: "encryptionkey@0"
I20260811 12:07:43.796761 6679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "113251304764470b92d34432d9a97af4"
format_stamp: "Formatted at 2026-08-11 12:07:43 on dist-test-slave-mjjr"
server_key: "b9bb963c8ca92287097adef534f93ac1"
server_key_iv: "a2d73f07211346fe21fcf0412f77c561"
server_key_version: "encryptionkey@0"
I20260811 12:07:43.801782 6679 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:07:43.805325 6702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:43.806666 6679 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 12:07:43.806864 6679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "113251304764470b92d34432d9a97af4"
format_stamp: "Formatted at 2026-08-11 12:07:43 on dist-test-slave-mjjr"
server_key: "b9bb963c8ca92287097adef534f93ac1"
server_key_iv: "a2d73f07211346fe21fcf0412f77c561"
server_key_version: "encryptionkey@0"
I20260811 12:07:43.807082 6679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:07:43.825098 6679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:07:43.825987 6679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:07:43.826283 6679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:07:43.845059 6679 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.190:44807
I20260811 12:07:43.845095 6788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.190:44807 every 8 connection(s)
I20260811 12:07:43.846560 6679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:07:43.850317 6789 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:07:43.854614 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 6679
I20260811 12:07:43.854807 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:07:43.855396 10102 external_mini_cluster.cc:1428] Setting key 9391bc16a68308ad2350f4df1ed310eb
I20260811 12:07:43.861963 6789 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 113251304764470b92d34432d9a97af4: Bootstrap starting.
I20260811 12:07:43.865350 6789 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 113251304764470b92d34432d9a97af4: Neither blocks nor log segments found. Creating new log.
I20260811 12:07:43.866605 6789 log.cc:824] T 00000000000000000000000000000000 P 113251304764470b92d34432d9a97af4: Log is configured to *not* fsync() on all Append() calls
I20260811 12:07:43.878539 6789 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 113251304764470b92d34432d9a97af4: No bootstrap required, opened a new log
I20260811 12:07:43.879447 6792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:43.857437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36948 (local address 127.9.221.190:44807)
0811 12:07:43.858433 (+ 996us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:43.858444 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:43.858488 (+ 44us) server_negotiation.cc:407] Connection header received
0811 12:07:43.860685 (+ 2197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:43.860706 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:43.861168 (+ 462us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:43.861572 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:43.862166 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:43.863483 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:43.865347 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:43.867147 (+ 1800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:43.867322 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:43.875127 (+ 7805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:43.875141 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:43.875164 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:07:43.876071 (+ 907us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:43.876195 (+ 124us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:07:43.876232 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:43.876379 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:43.877280 (+ 901us) server_negotiation.cc:299] Negotiation successful
0811 12:07:43.877731 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":333,"threads_started":1}
I20260811 12:07:43.891465 6789 raft_consensus.cc:348] T 00000000000000000000000000000000 P 113251304764470b92d34432d9a97af4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "113251304764470b92d34432d9a97af4" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 44807 } }
I20260811 12:07:43.892207 6789 raft_consensus.cc:374] T 00000000000000000000000000000000 P 113251304764470b92d34432d9a97af4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:07:43.892458 6789 raft_consensus.cc:729] T 00000000000000000000000000000000 P 113251304764470b92d34432d9a97af4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 113251304764470b92d34432d9a97af4, State: Initialized, Role: FOLLOWER
I20260811 12:07:43.893288 6789 consensus_queue.cc:260] T 00000000000000000000000000000000 P 113251304764470b92d34432d9a97af4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "113251304764470b92d34432d9a97af4" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 44807 } }
I20260811 12:07:43.893620 6789 raft_consensus.cc:388] T 00000000000000000000000000000000 P 113251304764470b92d34432d9a97af4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:07:43.893772 6789 raft_consensus.cc:482] T 00000000000000000000000000000000 P 113251304764470b92d34432d9a97af4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:07:43.893985 6789 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 113251304764470b92d34432d9a97af4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:07:43.897957 6789 raft_consensus.cc:504] T 00000000000000000000000000000000 P 113251304764470b92d34432d9a97af4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "113251304764470b92d34432d9a97af4" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 44807 } }
I20260811 12:07:43.898624 6789 leader_election.cc:302] T 00000000000000000000000000000000 P 113251304764470b92d34432d9a97af4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 113251304764470b92d34432d9a97af4; no voters:
I20260811 12:07:43.899361 6789 leader_election.cc:288] T 00000000000000000000000000000000 P 113251304764470b92d34432d9a97af4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:07:43.899523 6797 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 113251304764470b92d34432d9a97af4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:07:43.900367 6797 raft_consensus.cc:686] T 00000000000000000000000000000000 P 113251304764470b92d34432d9a97af4 [term 1 LEADER]: Becoming Leader. State: Replica: 113251304764470b92d34432d9a97af4, State: Running, Role: LEADER
I20260811 12:07:43.900952 6797 consensus_queue.cc:237] T 00000000000000000000000000000000 P 113251304764470b92d34432d9a97af4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "113251304764470b92d34432d9a97af4" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 44807 } }
I20260811 12:07:43.901897 6789 sys_catalog.cc:564] T 00000000000000000000000000000000 P 113251304764470b92d34432d9a97af4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:07:43.904139 6799 sys_catalog.cc:455] T 00000000000000000000000000000000 P 113251304764470b92d34432d9a97af4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 113251304764470b92d34432d9a97af4. Latest consensus state: current_term: 1 leader_uuid: "113251304764470b92d34432d9a97af4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "113251304764470b92d34432d9a97af4" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 44807 } } }
I20260811 12:07:43.904609 6799 sys_catalog.cc:458] T 00000000000000000000000000000000 P 113251304764470b92d34432d9a97af4 [sys.catalog]: This master's current role is: LEADER
I20260811 12:07:43.905486 6798 sys_catalog.cc:455] T 00000000000000000000000000000000 P 113251304764470b92d34432d9a97af4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "113251304764470b92d34432d9a97af4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "113251304764470b92d34432d9a97af4" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 44807 } } }
I20260811 12:07:43.905870 6798 sys_catalog.cc:458] T 00000000000000000000000000000000 P 113251304764470b92d34432d9a97af4 [sys.catalog]: This master's current role is: LEADER
I20260811 12:07:43.913321 6803 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:07:43.918895 6803 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:07:43.929167 6803 catalog_manager.cc:1347] Generated new cluster ID: 3bfeb1932a3444d9bbfebd6397e5f4fd
I20260811 12:07:43.929306 6803 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:07:43.946350 6803 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:07:43.948266 6803 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:07:43.957593 6803 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 113251304764470b92d34432d9a97af4: Generated new TSK 0
I20260811 12:07:43.958508 6803 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:07:43.975874 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.129:0
--local_ip_for_outbound_sockets=127.9.221.129
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:44807
--builtin_ntp_servers=127.9.221.148:35513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 12:07:44.333096 6825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:07:44.333381 6825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:07:44.333468 6825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:07:44.340576 6825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:07:44.340687 6825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:07:44.340794 6825 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.129
I20260811 12:07:44.350037 6825 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:35513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:44807
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.129
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:07:44.351353 6825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:07:44.353087 6825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:07:44.366385 6834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:44.366376 6836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:44.366948 6838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:07:44.367219 6825 server_base.cc:1034] running on GCE node
I20260811 12:07:44.367880 6825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:07:44.368816 6825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:07:44.370064 6825 hybrid_clock.cc:648] HybridClock initialized: now 1786450064370026 us; error 49 us; skew 500 ppm
I20260811 12:07:44.370388 6825 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:07:44.372587 6825 webserver.cc:460] Webserver started at http://127.9.221.129:39085/ using document root <none> and password file <none>
I20260811 12:07:44.373266 6825 fs_manager.cc:359] Metadata directory not provided
I20260811 12:07:44.373404 6825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:07:44.373728 6825 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:07:44.376637 6825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-0/data/instance:
uuid: "f7f6545eb43942e88fcc745d92f1d0c3"
format_stamp: "Formatted at 2026-08-11 12:07:44 on dist-test-slave-mjjr"
server_key: "cae40fac229ec3b80a833813f11c0553"
server_key_iv: "816493ba2034afd011661ce2996100d1"
server_key_version: "encryptionkey@0"
I20260811 12:07:44.377571 6825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance:
uuid: "f7f6545eb43942e88fcc745d92f1d0c3"
format_stamp: "Formatted at 2026-08-11 12:07:44 on dist-test-slave-mjjr"
server_key: "cae40fac229ec3b80a833813f11c0553"
server_key_iv: "816493ba2034afd011661ce2996100d1"
server_key_version: "encryptionkey@0"
I20260811 12:07:44.382989 6825 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260811 12:07:44.386325 6847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:44.387526 6825 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 12:07:44.387743 6825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-0/wal
uuid: "f7f6545eb43942e88fcc745d92f1d0c3"
format_stamp: "Formatted at 2026-08-11 12:07:44 on dist-test-slave-mjjr"
server_key: "cae40fac229ec3b80a833813f11c0553"
server_key_iv: "816493ba2034afd011661ce2996100d1"
server_key_version: "encryptionkey@0"
I20260811 12:07:44.387959 6825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:07:44.402134 6825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:07:44.402895 6825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:07:44.403199 6825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:07:44.404196 6825 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:07:44.405586 6825 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:07:44.405707 6825 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:44.405845 6825 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:07:44.405933 6825 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:44.439486 6825 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.129:45201
I20260811 12:07:44.439535 7028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.129:45201 every 8 connection(s)
I20260811 12:07:44.440863 6825 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
I20260811 12:07:44.442251 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 6825
I20260811 12:07:44.442442 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance
I20260811 12:07:44.442924 10102 external_mini_cluster.cc:1428] Setting key e0ce258608b4e99220a91239db362f79
I20260811 12:07:44.446063 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.130:0
--local_ip_for_outbound_sockets=127.9.221.130
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:44807
--builtin_ntp_servers=127.9.221.148:35513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 12:07:44.460271 7032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:44.443830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:34159 (local address 127.9.221.190:44807)
0811 12:07:44.448084 (+ 4254us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:44.448093 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:44.449157 (+ 1064us) server_negotiation.cc:407] Connection header received
0811 12:07:44.452038 (+ 2881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:44.452042 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:44.452107 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:44.452201 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:44.453755 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:44.454486 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:44.457178 (+ 2692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:44.457628 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:44.457783 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:44.459165 (+ 1382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:44.459175 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:44.459182 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:07:44.459208 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:44.459276 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:07:44.459294 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:44.459393 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:44.459674 (+ 281us) server_negotiation.cc:299] Negotiation successful
0811 12:07:44.459785 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4082,"thread_start_us":3950,"threads_started":1}
I20260811 12:07:44.461284 7037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:44.444518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44807 (local address 127.9.221.129:34159)
0811 12:07:44.448979 (+ 4461us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:44.449030 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 12:07:44.451747 (+ 2717us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:07:44.452516 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:07:44.452530 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:07:44.452655 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:07:44.453545 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:44.453574 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:44.454650 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:44.454660 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:44.457002 (+ 2342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:44.457023 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:44.457868 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:44.457876 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:44.458114 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:44.458847 (+ 733us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:07:44.458850 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:07:44.458889 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:07:44.458965 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:07:44.459415 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:07:44.459425 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:07:44.459432 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 12:07:44.459600 (+ 168us) client_negotiation.cc:240] Negotiation successful
0811 12:07:44.460084 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3858,"thread_start_us":3593,"threads_started":1}
I20260811 12:07:44.463099 7029 heartbeater.cc:347] Connected to a master server at 127.9.221.190:44807
I20260811 12:07:44.463438 7029 heartbeater.cc:464] Registering TS with master...
I20260811 12:07:44.464306 7029 heartbeater.cc:511] Master 127.9.221.190:44807 requested a full tablet report, sending...
I20260811 12:07:44.466486 6724 ts_manager.cc:194] Registered new tserver with Master: f7f6545eb43942e88fcc745d92f1d0c3 (127.9.221.129:45201)
I20260811 12:07:44.468060 6724 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.221.129:34159
W20260811 12:07:44.784407 7038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:07:44.784824 7038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:07:44.784999 7038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:07:44.792291 7038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:07:44.792433 7038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:07:44.792618 7038 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.130
I20260811 12:07:44.801569 7038 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:35513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:44807
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.130
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:07:44.803064 7038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:07:44.804961 7038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:07:44.818702 7044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:44.818743 7048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:44.818765 7045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:07:44.820031 7038 server_base.cc:1034] running on GCE node
I20260811 12:07:44.820619 7038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:07:44.821524 7038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:07:44.822723 7038 hybrid_clock.cc:648] HybridClock initialized: now 1786450064822647 us; error 67 us; skew 500 ppm
I20260811 12:07:44.823007 7038 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:07:44.825258 7038 webserver.cc:460] Webserver started at http://127.9.221.130:38949/ using document root <none> and password file <none>
I20260811 12:07:44.825912 7038 fs_manager.cc:359] Metadata directory not provided
I20260811 12:07:44.826001 7038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:07:44.826259 7038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:07:44.829267 7038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-1/data/instance:
uuid: "cbd8237e65464ef5883023baee5d6b66"
format_stamp: "Formatted at 2026-08-11 12:07:44 on dist-test-slave-mjjr"
server_key: "64fdd7ecd2d71fc01996c3d1adca9525"
server_key_iv: "f46b3b29fa3f610109568436a60045f3"
server_key_version: "encryptionkey@0"
I20260811 12:07:44.829998 7038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance:
uuid: "cbd8237e65464ef5883023baee5d6b66"
format_stamp: "Formatted at 2026-08-11 12:07:44 on dist-test-slave-mjjr"
server_key: "64fdd7ecd2d71fc01996c3d1adca9525"
server_key_iv: "f46b3b29fa3f610109568436a60045f3"
server_key_version: "encryptionkey@0"
I20260811 12:07:44.834903 7038 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:07:44.838199 7058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:44.839500 7038 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:07:44.839687 7038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-1/wal
uuid: "cbd8237e65464ef5883023baee5d6b66"
format_stamp: "Formatted at 2026-08-11 12:07:44 on dist-test-slave-mjjr"
server_key: "64fdd7ecd2d71fc01996c3d1adca9525"
server_key_iv: "f46b3b29fa3f610109568436a60045f3"
server_key_version: "encryptionkey@0"
I20260811 12:07:44.839886 7038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:07:44.858358 7038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:07:44.859184 7038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:07:44.859494 7038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:07:44.860518 7038 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:07:44.861981 7038 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:07:44.862098 7038 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:44.862231 7038 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:07:44.862326 7038 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:44.896258 7038 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.130:33709
I20260811 12:07:44.896304 7255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.130:33709 every 8 connection(s)
I20260811 12:07:44.897579 7038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
I20260811 12:07:44.899243 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 7038
I20260811 12:07:44.899431 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance
I20260811 12:07:44.900074 10102 external_mini_cluster.cc:1428] Setting key 4ed7fdc6f8fd35ea33bce9fb87e0bf0f
I20260811 12:07:44.903353 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.131:0
--local_ip_for_outbound_sockets=127.9.221.131
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:44807
--builtin_ntp_servers=127.9.221.148:35513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 12:07:44.910575 7032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:44.900451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:58837 (local address 127.9.221.190:44807)
0811 12:07:44.900647 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:44.900656 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:44.902176 (+ 1520us) server_negotiation.cc:407] Connection header received
0811 12:07:44.904239 (+ 2063us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:44.904244 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:44.904338 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:44.904446 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:44.905534 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:44.906245 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:44.907844 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:44.908240 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:44.908390 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:44.909734 (+ 1344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:44.909740 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:44.909743 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:07:44.909760 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:44.909792 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:07:44.909804 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:44.909905 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:44.910120 (+ 215us) server_negotiation.cc:299] Negotiation successful
0811 12:07:44.910218 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 12:07:44.911597 7259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:44.901012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44807 (local address 127.9.221.130:58837)
0811 12:07:44.902034 (+ 1022us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:44.902081 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 12:07:44.903989 (+ 1908us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:07:44.904730 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:07:44.904738 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:07:44.904832 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:07:44.905372 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:44.905387 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:44.906399 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:44.906405 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:44.907634 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:44.907700 (+ 66us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:44.908472 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:44.908479 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:44.908710 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:44.909424 (+ 714us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:07:44.909426 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:07:44.909463 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:07:44.909540 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:07:44.909904 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:07:44.909917 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:07:44.909927 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 12:07:44.910107 (+ 180us) client_negotiation.cc:240] Negotiation successful
0811 12:07:44.910496 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":336,"threads_started":1}
I20260811 12:07:44.913074 7256 heartbeater.cc:347] Connected to a master server at 127.9.221.190:44807
I20260811 12:07:44.913390 7256 heartbeater.cc:464] Registering TS with master...
I20260811 12:07:44.914207 7256 heartbeater.cc:511] Master 127.9.221.190:44807 requested a full tablet report, sending...
I20260811 12:07:44.915700 6724 ts_manager.cc:194] Registered new tserver with Master: cbd8237e65464ef5883023baee5d6b66 (127.9.221.130:33709)
I20260811 12:07:44.916930 6724 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.221.130:58837
W20260811 12:07:45.243997 7260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:07:45.244297 7260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:07:45.244418 7260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:07:45.251524 7260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:07:45.251610 7260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:07:45.251781 7260 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.131
I20260811 12:07:45.260561 7260 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:35513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:44807
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.131
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:07:45.261780 7260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:07:45.263495 7260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:07:45.277128 7267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:45.277114 7269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:45.277573 7272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:07:45.277858 7260 server_base.cc:1034] running on GCE node
I20260811 12:07:45.278486 7260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:07:45.279433 7260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:07:45.280664 7260 hybrid_clock.cc:648] HybridClock initialized: now 1786450065280598 us; error 53 us; skew 500 ppm
I20260811 12:07:45.280989 7260 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:07:45.283504 7260 webserver.cc:460] Webserver started at http://127.9.221.131:45219/ using document root <none> and password file <none>
I20260811 12:07:45.284245 7260 fs_manager.cc:359] Metadata directory not provided
I20260811 12:07:45.284376 7260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:07:45.284721 7260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:07:45.287710 7260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-2/data/instance:
uuid: "e1bb28ca0e0f40a29165f730a8f48eb9"
format_stamp: "Formatted at 2026-08-11 12:07:45 on dist-test-slave-mjjr"
server_key: "c9db592613953c3f7d95b09c10cee21e"
server_key_iv: "ed7fae820fe6d168b49ed72ac20933ed"
server_key_version: "encryptionkey@0"
I20260811 12:07:45.288522 7260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance:
uuid: "e1bb28ca0e0f40a29165f730a8f48eb9"
format_stamp: "Formatted at 2026-08-11 12:07:45 on dist-test-slave-mjjr"
server_key: "c9db592613953c3f7d95b09c10cee21e"
server_key_iv: "ed7fae820fe6d168b49ed72ac20933ed"
server_key_version: "encryptionkey@0"
I20260811 12:07:45.293531 7260 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260811 12:07:45.296904 7278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:45.298247 7260 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:07:45.298421 7260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-2/wal
uuid: "e1bb28ca0e0f40a29165f730a8f48eb9"
format_stamp: "Formatted at 2026-08-11 12:07:45 on dist-test-slave-mjjr"
server_key: "c9db592613953c3f7d95b09c10cee21e"
server_key_iv: "ed7fae820fe6d168b49ed72ac20933ed"
server_key_version: "encryptionkey@0"
I20260811 12:07:45.298632 7260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:07:45.331977 7260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:07:45.332854 7260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:07:45.333182 7260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:07:45.334208 7260 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:07:45.335683 7260 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:07:45.335811 7260 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:45.335949 7260 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:07:45.336040 7260 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:45.370528 7260 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.131:33379
I20260811 12:07:45.370618 7460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.131:33379 every 8 connection(s)
I20260811 12:07:45.371973 7260 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
I20260811 12:07:45.375123 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 7260
I20260811 12:07:45.375347 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance
I20260811 12:07:45.375969 10102 external_mini_cluster.cc:1428] Setting key e3f1730c39bf161557bf9ab63ae4c834
I20260811 12:07:45.385375 7032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:45.374552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:46659 (local address 127.9.221.190:44807)
0811 12:07:45.374723 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:45.374727 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:45.376705 (+ 1978us) server_negotiation.cc:407] Connection header received
0811 12:07:45.378953 (+ 2248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:45.378958 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:45.379046 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:45.379159 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:45.380349 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:45.381079 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:45.382618 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:45.383040 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:45.383179 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:45.384529 (+ 1350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:45.384539 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:45.384543 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:07:45.384559 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:45.384588 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:07:45.384600 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:45.384695 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:45.384943 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 12:07:45.385039 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 12:07:45.386343 7464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:45.374997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:44807 (local address 127.9.221.131:46659)
0811 12:07:45.376505 (+ 1508us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:45.376566 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 12:07:45.378662 (+ 2096us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:07:45.379389 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:07:45.379397 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:07:45.379476 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:07:45.380171 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:45.380185 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:45.381240 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:45.381244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:45.382464 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:45.382480 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:45.383267 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:45.383274 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:45.383503 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:45.384221 (+ 718us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:07:45.384224 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:07:45.384260 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:07:45.384331 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:07:45.384695 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:07:45.384708 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:07:45.384718 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 12:07:45.384915 (+ 197us) client_negotiation.cc:240] Negotiation successful
0811 12:07:45.385282 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":973,"thread_start_us":768,"threads_started":1}
I20260811 12:07:45.387573 7461 heartbeater.cc:347] Connected to a master server at 127.9.221.190:44807
I20260811 12:07:45.387904 7461 heartbeater.cc:464] Registering TS with master...
I20260811 12:07:45.388594 7461 heartbeater.cc:511] Master 127.9.221.190:44807 requested a full tablet report, sending...
I20260811 12:07:45.390054 6724 ts_manager.cc:194] Registered new tserver with Master: e1bb28ca0e0f40a29165f730a8f48eb9 (127.9.221.131:33379)
I20260811 12:07:45.391162 6724 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.221.131:46659
I20260811 12:07:45.391403 10102 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:07:45.408088 7032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:45.397697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36950 (local address 127.9.221.190:44807)
0811 12:07:45.397908 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:45.397914 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:45.400289 (+ 2375us) server_negotiation.cc:407] Connection header received
0811 12:07:45.400514 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:45.400518 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:45.400576 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:45.400700 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:45.401342 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:45.402374 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:45.405193 (+ 2819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:45.405795 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:45.405936 (+ 141us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:45.407263 (+ 1327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:45.407270 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:45.407274 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:07:45.407300 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:45.407333 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:07:45.407346 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:45.407497 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:45.407718 (+ 221us) server_negotiation.cc:299] Negotiation successful
0811 12:07:45.407796 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 12:07:45.412002 6724 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36950:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:07:45.419010 6724 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:07:45.454213 7480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:45.431078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44174 (local address 127.9.221.131:33379)
0811 12:07:45.443982 (+ 12904us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:45.444001 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:45.444046 (+ 45us) server_negotiation.cc:407] Connection header received
0811 12:07:45.444571 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:45.444584 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:45.444915 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:45.446137 (+ 1222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:07:45.446509 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:45.448613 (+ 2104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:45.451619 (+ 3006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:45.452922 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:45.453084 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:45.453366 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:45.453644 (+ 278us) server_negotiation.cc:299] Negotiation successful
0811 12:07:45.453862 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11549,"thread_start_us":11385,"threads_started":1}
I20260811 12:07:45.454978 7484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:45.430761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.131:33379 (local address 127.0.0.1:44174)
0811 12:07:45.443909 (+ 13148us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:45.443953 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 12:07:45.444129 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:07:45.445200 (+ 1071us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:07:45.445211 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:07:45.445255 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:07:45.445866 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:45.445884 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:45.448981 (+ 3097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:45.448995 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:45.451367 (+ 2372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:45.451401 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:45.453119 (+ 1718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:45.453126 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:45.453343 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:45.453404 (+ 61us) client_negotiation.cc:769] Sending connection context
0811 12:07:45.453528 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 12:07:45.454591 (+ 1063us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12890,"mutex_wait_us":870,"thread_start_us":17630,"threads_started":1}
I20260811 12:07:45.458374 7476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:45.430957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.130:33709 (local address 127.0.0.1:47034)
0811 12:07:45.446908 (+ 15951us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:45.446928 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:07:45.447042 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:07:45.448611 (+ 1569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:07:45.448616 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:07:45.448640 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:07:45.451462 (+ 2822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:45.451475 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:45.453269 (+ 1794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:45.453275 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:45.456078 (+ 2803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:45.456093 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:45.457590 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:45.457597 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:45.457759 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:45.457792 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 12:07:45.457880 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 12:07:45.458005 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15814,"mutex_wait_us":6,"thread_start_us":18418,"threads_started":1}
I20260811 12:07:45.458503 7481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:45.431240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47034 (local address 127.9.221.130:33709)
0811 12:07:45.443485 (+ 12245us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:45.443495 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:45.447155 (+ 3660us) server_negotiation.cc:407] Connection header received
0811 12:07:45.447258 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:45.447268 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:45.447505 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:45.447700 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:07:45.451710 (+ 4010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:45.453097 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:45.456318 (+ 3221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:45.457372 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:45.457521 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:45.457754 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:45.457970 (+ 216us) server_negotiation.cc:299] Negotiation successful
0811 12:07:45.458144 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11741,"thread_start_us":11623,"threads_started":1}
I20260811 12:07:45.459959 7479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:45.433060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:45201 (local address 127.0.0.1:39158)
0811 12:07:45.448188 (+ 15128us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:45.448205 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:07:45.448310 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:07:45.452098 (+ 3788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:07:45.452103 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:07:45.452128 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:07:45.452503 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:45.452515 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:45.454848 (+ 2333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:45.454852 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:45.456928 (+ 2076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:45.456944 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:45.459053 (+ 2109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:45.459060 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:45.459236 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:45.459278 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 12:07:45.459402 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 12:07:45.459552 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15013,"mutex_wait_us":4,"thread_start_us":10406,"threads_started":1}
I20260811 12:07:45.459316 7362 tablet_service.cc:1467] Processing CreateTablet for tablet c6af868c1b6a43eea0c212a3c919bb7c (DEFAULT_TABLE table=test-table [id=2e8c6fe10b25429ea8483e37b36ad602]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:07:45.461179 7485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:45.433690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39158 (local address 127.9.221.129:45201)
0811 12:07:45.451492 (+ 17802us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:45.451501 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:45.451530 (+ 29us) server_negotiation.cc:407] Connection header received
0811 12:07:45.451634 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:45.451640 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:45.451873 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:45.452079 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:07:45.452692 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:45.454704 (+ 2012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:45.457142 (+ 2438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:45.458829 (+ 1687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:45.458985 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:45.459239 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:45.459495 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 12:07:45.460850 (+ 1355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17625,"thread_start_us":17503,"threads_started":1}
I20260811 12:07:45.461356 7362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6af868c1b6a43eea0c212a3c919bb7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:07:45.462317 7147 tablet_service.cc:1467] Processing CreateTablet for tablet c6af868c1b6a43eea0c212a3c919bb7c (DEFAULT_TABLE table=test-table [id=2e8c6fe10b25429ea8483e37b36ad602]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:07:45.463393 7147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6af868c1b6a43eea0c212a3c919bb7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:07:45.464699 6931 tablet_service.cc:1467] Processing CreateTablet for tablet c6af868c1b6a43eea0c212a3c919bb7c (DEFAULT_TABLE table=test-table [id=2e8c6fe10b25429ea8483e37b36ad602]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:07:45.465979 6931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6af868c1b6a43eea0c212a3c919bb7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:07:45.471261 7029 heartbeater.cc:503] Master 127.9.221.190:44807 was elected leader, sending a full tablet report...
I20260811 12:07:45.475338 7486 tablet_bootstrap.cc:492] T c6af868c1b6a43eea0c212a3c919bb7c P cbd8237e65464ef5883023baee5d6b66: Bootstrap starting.
I20260811 12:07:45.477382 7489 tablet_bootstrap.cc:492] T c6af868c1b6a43eea0c212a3c919bb7c P e1bb28ca0e0f40a29165f730a8f48eb9: Bootstrap starting.
I20260811 12:07:45.480249 7490 tablet_bootstrap.cc:492] T c6af868c1b6a43eea0c212a3c919bb7c P f7f6545eb43942e88fcc745d92f1d0c3: Bootstrap starting.
I20260811 12:07:45.480650 7486 tablet_bootstrap.cc:654] T c6af868c1b6a43eea0c212a3c919bb7c P cbd8237e65464ef5883023baee5d6b66: Neither blocks nor log segments found. Creating new log.
I20260811 12:07:45.481058 7489 tablet_bootstrap.cc:654] T c6af868c1b6a43eea0c212a3c919bb7c P e1bb28ca0e0f40a29165f730a8f48eb9: Neither blocks nor log segments found. Creating new log.
I20260811 12:07:45.482008 7486 log.cc:824] T c6af868c1b6a43eea0c212a3c919bb7c P cbd8237e65464ef5883023baee5d6b66: Log is configured to *not* fsync() on all Append() calls
I20260811 12:07:45.482246 7489 log.cc:824] T c6af868c1b6a43eea0c212a3c919bb7c P e1bb28ca0e0f40a29165f730a8f48eb9: Log is configured to *not* fsync() on all Append() calls
I20260811 12:07:45.483461 7490 tablet_bootstrap.cc:654] T c6af868c1b6a43eea0c212a3c919bb7c P f7f6545eb43942e88fcc745d92f1d0c3: Neither blocks nor log segments found. Creating new log.
I20260811 12:07:45.484447 7490 log.cc:824] T c6af868c1b6a43eea0c212a3c919bb7c P f7f6545eb43942e88fcc745d92f1d0c3: Log is configured to *not* fsync() on all Append() calls
I20260811 12:07:45.485345 7486 tablet_bootstrap.cc:492] T c6af868c1b6a43eea0c212a3c919bb7c P cbd8237e65464ef5883023baee5d6b66: No bootstrap required, opened a new log
I20260811 12:07:45.485342 7489 tablet_bootstrap.cc:492] T c6af868c1b6a43eea0c212a3c919bb7c P e1bb28ca0e0f40a29165f730a8f48eb9: No bootstrap required, opened a new log
I20260811 12:07:45.485626 7486 ts_tablet_manager.cc:1397] T c6af868c1b6a43eea0c212a3c919bb7c P cbd8237e65464ef5883023baee5d6b66: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.003s
I20260811 12:07:45.485626 7489 ts_tablet_manager.cc:1397] T c6af868c1b6a43eea0c212a3c919bb7c P e1bb28ca0e0f40a29165f730a8f48eb9: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 12:07:45.487008 7490 tablet_bootstrap.cc:492] T c6af868c1b6a43eea0c212a3c919bb7c P f7f6545eb43942e88fcc745d92f1d0c3: No bootstrap required, opened a new log
I20260811 12:07:45.487304 7490 ts_tablet_manager.cc:1397] T c6af868c1b6a43eea0c212a3c919bb7c P f7f6545eb43942e88fcc745d92f1d0c3: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.003s
I20260811 12:07:45.495052 7490 raft_consensus.cc:348] T c6af868c1b6a43eea0c212a3c919bb7c P f7f6545eb43942e88fcc745d92f1d0c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1bb28ca0e0f40a29165f730a8f48eb9" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 33379 } } peers { permanent_uuid: "cbd8237e65464ef5883023baee5d6b66" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 33709 } } peers { permanent_uuid: "f7f6545eb43942e88fcc745d92f1d0c3" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 45201 } }
I20260811 12:07:45.495462 7490 raft_consensus.cc:374] T c6af868c1b6a43eea0c212a3c919bb7c P f7f6545eb43942e88fcc745d92f1d0c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:07:45.495577 7490 raft_consensus.cc:729] T c6af868c1b6a43eea0c212a3c919bb7c P f7f6545eb43942e88fcc745d92f1d0c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7f6545eb43942e88fcc745d92f1d0c3, State: Initialized, Role: FOLLOWER
I20260811 12:07:45.496263 7490 consensus_queue.cc:260] T c6af868c1b6a43eea0c212a3c919bb7c P f7f6545eb43942e88fcc745d92f1d0c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1bb28ca0e0f40a29165f730a8f48eb9" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 33379 } } peers { permanent_uuid: "cbd8237e65464ef5883023baee5d6b66" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 33709 } } peers { permanent_uuid: "f7f6545eb43942e88fcc745d92f1d0c3" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 45201 } }
I20260811 12:07:45.496490 7486 raft_consensus.cc:348] T c6af868c1b6a43eea0c212a3c919bb7c P cbd8237e65464ef5883023baee5d6b66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1bb28ca0e0f40a29165f730a8f48eb9" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 33379 } } peers { permanent_uuid: "cbd8237e65464ef5883023baee5d6b66" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 33709 } } peers { permanent_uuid: "f7f6545eb43942e88fcc745d92f1d0c3" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 45201 } }
I20260811 12:07:45.497013 7486 raft_consensus.cc:374] T c6af868c1b6a43eea0c212a3c919bb7c P cbd8237e65464ef5883023baee5d6b66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:07:45.496855 7489 raft_consensus.cc:348] T c6af868c1b6a43eea0c212a3c919bb7c P e1bb28ca0e0f40a29165f730a8f48eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1bb28ca0e0f40a29165f730a8f48eb9" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 33379 } } peers { permanent_uuid: "cbd8237e65464ef5883023baee5d6b66" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 33709 } } peers { permanent_uuid: "f7f6545eb43942e88fcc745d92f1d0c3" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 45201 } }
I20260811 12:07:45.497159 7486 raft_consensus.cc:729] T c6af868c1b6a43eea0c212a3c919bb7c P cbd8237e65464ef5883023baee5d6b66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbd8237e65464ef5883023baee5d6b66, State: Initialized, Role: FOLLOWER
I20260811 12:07:45.497327 7489 raft_consensus.cc:374] T c6af868c1b6a43eea0c212a3c919bb7c P e1bb28ca0e0f40a29165f730a8f48eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:07:45.497474 7489 raft_consensus.cc:729] T c6af868c1b6a43eea0c212a3c919bb7c P e1bb28ca0e0f40a29165f730a8f48eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1bb28ca0e0f40a29165f730a8f48eb9, State: Initialized, Role: FOLLOWER
I20260811 12:07:45.498103 7490 ts_tablet_manager.cc:1428] T c6af868c1b6a43eea0c212a3c919bb7c P f7f6545eb43942e88fcc745d92f1d0c3: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260811 12:07:45.497977 7486 consensus_queue.cc:260] T c6af868c1b6a43eea0c212a3c919bb7c P cbd8237e65464ef5883023baee5d6b66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1bb28ca0e0f40a29165f730a8f48eb9" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 33379 } } peers { permanent_uuid: "cbd8237e65464ef5883023baee5d6b66" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 33709 } } peers { permanent_uuid: "f7f6545eb43942e88fcc745d92f1d0c3" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 45201 } }
I20260811 12:07:45.498173 7489 consensus_queue.cc:260] T c6af868c1b6a43eea0c212a3c919bb7c P e1bb28ca0e0f40a29165f730a8f48eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1bb28ca0e0f40a29165f730a8f48eb9" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 33379 } } peers { permanent_uuid: "cbd8237e65464ef5883023baee5d6b66" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 33709 } } peers { permanent_uuid: "f7f6545eb43942e88fcc745d92f1d0c3" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 45201 } }
I20260811 12:07:45.499277 7256 heartbeater.cc:503] Master 127.9.221.190:44807 was elected leader, sending a full tablet report...
I20260811 12:07:45.500140 7486 ts_tablet_manager.cc:1428] T c6af868c1b6a43eea0c212a3c919bb7c P cbd8237e65464ef5883023baee5d6b66: Time spent starting tablet: real 0.014s user 0.014s sys 0.001s
I20260811 12:07:45.501062 7461 heartbeater.cc:503] Master 127.9.221.190:44807 was elected leader, sending a full tablet report...
I20260811 12:07:45.501672 7489 ts_tablet_manager.cc:1428] T c6af868c1b6a43eea0c212a3c919bb7c P e1bb28ca0e0f40a29165f730a8f48eb9: Time spent starting tablet: real 0.016s user 0.006s sys 0.010s
W20260811 12:07:45.623926 7463 tablet.cc:2369] T c6af868c1b6a43eea0c212a3c919bb7c P e1bb28ca0e0f40a29165f730a8f48eb9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:07:45.650527 7257 tablet.cc:2369] T c6af868c1b6a43eea0c212a3c919bb7c P cbd8237e65464ef5883023baee5d6b66: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:07:45.652860 7499 raft_consensus.cc:482] T c6af868c1b6a43eea0c212a3c919bb7c P f7f6545eb43942e88fcc745d92f1d0c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:07:45.653201 7499 raft_consensus.cc:504] T c6af868c1b6a43eea0c212a3c919bb7c P f7f6545eb43942e88fcc745d92f1d0c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1bb28ca0e0f40a29165f730a8f48eb9" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 33379 } } peers { permanent_uuid: "cbd8237e65464ef5883023baee5d6b66" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 33709 } } peers { permanent_uuid: "f7f6545eb43942e88fcc745d92f1d0c3" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 45201 } }
I20260811 12:07:45.655056 7499 leader_election.cc:288] T c6af868c1b6a43eea0c212a3c919bb7c P f7f6545eb43942e88fcc745d92f1d0c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1bb28ca0e0f40a29165f730a8f48eb9 (127.9.221.131:33379), cbd8237e65464ef5883023baee5d6b66 (127.9.221.130:33709)
I20260811 12:07:45.662214 7480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:45.655459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:36315 (local address 127.9.221.131:33379)
0811 12:07:45.655708 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:45.655716 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:45.656577 (+ 861us) server_negotiation.cc:407] Connection header received
0811 12:07:45.656808 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:45.656818 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:45.656881 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:45.656982 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:07:45.657579 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:45.658331 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:45.660434 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:45.661233 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:45.661427 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:45.661495 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:45.661807 (+ 312us) server_negotiation.cc:299] Negotiation successful
0811 12:07:45.661955 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 12:07:45.662232 7481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:45.655756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:47715 (local address 127.9.221.130:33709)
0811 12:07:45.655981 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:45.655990 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:45.656754 (+ 764us) server_negotiation.cc:407] Connection header received
0811 12:07:45.656983 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:45.656987 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:45.657060 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:45.657158 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:07:45.657775 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:45.658450 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:45.660494 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:45.661244 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:45.661407 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:45.661487 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:45.661807 (+ 320us) server_negotiation.cc:299] Negotiation successful
0811 12:07:45.661955 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 12:07:45.662405 7502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:45.655507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.131:33379 (local address 127.9.221.129:36315)
0811 12:07:45.656441 (+ 934us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:45.656464 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:07:45.656653 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:07:45.657009 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:07:45.657013 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:07:45.657037 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:07:45.657424 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:45.657433 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:45.658462 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:45.658470 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:45.660230 (+ 1760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:45.660245 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:45.661440 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:45.661447 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:45.661629 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:45.661670 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 12:07:45.661777 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 12:07:45.661971 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":802,"mutex_wait_us":154,"thread_start_us":487,"threads_started":1}
I20260811 12:07:45.662432 7505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:45.655602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.130:33709 (local address 127.9.221.129:47715)
0811 12:07:45.656644 (+ 1042us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:45.656658 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 12:07:45.656813 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:07:45.657181 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:07:45.657185 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:07:45.657210 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:07:45.657608 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:45.657616 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:45.658591 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:45.658600 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:45.660278 (+ 1678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:45.660293 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:45.661460 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:45.661467 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:45.661630 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:45.661674 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 12:07:45.661777 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 12:07:45.661978 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":927,"mutex_wait_us":61,"thread_start_us":692,"threads_started":1}
I20260811 12:07:45.663602 7389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6af868c1b6a43eea0c212a3c919bb7c" candidate_uuid: "f7f6545eb43942e88fcc745d92f1d0c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1bb28ca0e0f40a29165f730a8f48eb9" is_pre_election: true
I20260811 12:07:45.663599 7178 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6af868c1b6a43eea0c212a3c919bb7c" candidate_uuid: "f7f6545eb43942e88fcc745d92f1d0c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbd8237e65464ef5883023baee5d6b66" is_pre_election: true
I20260811 12:07:45.664170 7389 raft_consensus.cc:2454] T c6af868c1b6a43eea0c212a3c919bb7c P e1bb28ca0e0f40a29165f730a8f48eb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7f6545eb43942e88fcc745d92f1d0c3 in term 0.
I20260811 12:07:45.664170 7178 raft_consensus.cc:2454] T c6af868c1b6a43eea0c212a3c919bb7c P cbd8237e65464ef5883023baee5d6b66 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7f6545eb43942e88fcc745d92f1d0c3 in term 0.
I20260811 12:07:45.664858 6853 leader_election.cc:302] T c6af868c1b6a43eea0c212a3c919bb7c P f7f6545eb43942e88fcc745d92f1d0c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e1bb28ca0e0f40a29165f730a8f48eb9, f7f6545eb43942e88fcc745d92f1d0c3; no voters:
I20260811 12:07:45.665261 7499 raft_consensus.cc:2781] T c6af868c1b6a43eea0c212a3c919bb7c P f7f6545eb43942e88fcc745d92f1d0c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:07:45.665418 7499 raft_consensus.cc:482] T c6af868c1b6a43eea0c212a3c919bb7c P f7f6545eb43942e88fcc745d92f1d0c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:07:45.665529 7499 raft_consensus.cc:3037] T c6af868c1b6a43eea0c212a3c919bb7c P f7f6545eb43942e88fcc745d92f1d0c3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:07:45.667745 7499 raft_consensus.cc:504] T c6af868c1b6a43eea0c212a3c919bb7c P f7f6545eb43942e88fcc745d92f1d0c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1bb28ca0e0f40a29165f730a8f48eb9" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 33379 } } peers { permanent_uuid: "cbd8237e65464ef5883023baee5d6b66" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 33709 } } peers { permanent_uuid: "f7f6545eb43942e88fcc745d92f1d0c3" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 45201 } }
I20260811 12:07:45.668401 7499 leader_election.cc:288] T c6af868c1b6a43eea0c212a3c919bb7c P f7f6545eb43942e88fcc745d92f1d0c3 [CANDIDATE]: Term 1 election: Requested vote from peers e1bb28ca0e0f40a29165f730a8f48eb9 (127.9.221.131:33379), cbd8237e65464ef5883023baee5d6b66 (127.9.221.130:33709)
I20260811 12:07:45.668776 7389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6af868c1b6a43eea0c212a3c919bb7c" candidate_uuid: "f7f6545eb43942e88fcc745d92f1d0c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1bb28ca0e0f40a29165f730a8f48eb9"
I20260811 12:07:45.668874 7178 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6af868c1b6a43eea0c212a3c919bb7c" candidate_uuid: "f7f6545eb43942e88fcc745d92f1d0c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbd8237e65464ef5883023baee5d6b66"
I20260811 12:07:45.668979 7389 raft_consensus.cc:3037] T c6af868c1b6a43eea0c212a3c919bb7c P e1bb28ca0e0f40a29165f730a8f48eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:07:45.669106 7178 raft_consensus.cc:3037] T c6af868c1b6a43eea0c212a3c919bb7c P cbd8237e65464ef5883023baee5d6b66 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:07:45.671208 7389 raft_consensus.cc:2454] T c6af868c1b6a43eea0c212a3c919bb7c P e1bb28ca0e0f40a29165f730a8f48eb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7f6545eb43942e88fcc745d92f1d0c3 in term 1.
I20260811 12:07:45.671442 7178 raft_consensus.cc:2454] T c6af868c1b6a43eea0c212a3c919bb7c P cbd8237e65464ef5883023baee5d6b66 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7f6545eb43942e88fcc745d92f1d0c3 in term 1.
I20260811 12:07:45.671680 6853 leader_election.cc:302] T c6af868c1b6a43eea0c212a3c919bb7c P f7f6545eb43942e88fcc745d92f1d0c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e1bb28ca0e0f40a29165f730a8f48eb9, f7f6545eb43942e88fcc745d92f1d0c3; no voters:
I20260811 12:07:45.671978 7499 raft_consensus.cc:2781] T c6af868c1b6a43eea0c212a3c919bb7c P f7f6545eb43942e88fcc745d92f1d0c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:07:45.672449 7499 raft_consensus.cc:686] T c6af868c1b6a43eea0c212a3c919bb7c P f7f6545eb43942e88fcc745d92f1d0c3 [term 1 LEADER]: Becoming Leader. State: Replica: f7f6545eb43942e88fcc745d92f1d0c3, State: Running, Role: LEADER
I20260811 12:07:45.672917 7499 consensus_queue.cc:237] T c6af868c1b6a43eea0c212a3c919bb7c P f7f6545eb43942e88fcc745d92f1d0c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1bb28ca0e0f40a29165f730a8f48eb9" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 33379 } } peers { permanent_uuid: "cbd8237e65464ef5883023baee5d6b66" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 33709 } } peers { permanent_uuid: "f7f6545eb43942e88fcc745d92f1d0c3" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 45201 } }
I20260811 12:07:45.677960 6722 catalog_manager.cc:5495] T c6af868c1b6a43eea0c212a3c919bb7c P f7f6545eb43942e88fcc745d92f1d0c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to f7f6545eb43942e88fcc745d92f1d0c3 (127.9.221.129). New cstate: current_term: 1 leader_uuid: "f7f6545eb43942e88fcc745d92f1d0c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1bb28ca0e0f40a29165f730a8f48eb9" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 33379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cbd8237e65464ef5883023baee5d6b66" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 33709 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7f6545eb43942e88fcc745d92f1d0c3" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 45201 } health_report { overall_health: HEALTHY } } }
W20260811 12:07:45.693516 7030 tablet.cc:2369] T c6af868c1b6a43eea0c212a3c919bb7c P f7f6545eb43942e88fcc745d92f1d0c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:07:45.748281 7485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:45.742623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47280 (local address 127.9.221.129:45201)
0811 12:07:45.742838 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:45.742844 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:45.742892 (+ 48us) server_negotiation.cc:407] Connection header received
0811 12:07:45.743014 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:45.743022 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:45.743133 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:45.743300 (+ 167us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:07:45.743799 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:45.744485 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:45.745898 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:45.746329 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:45.746487 (+ 158us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:45.747128 (+ 641us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:07:45.747623 (+ 495us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:07:45.747855 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:45.748020 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 12:07:45.748066 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 12:07:45.756386 7178 raft_consensus.cc:1261] T c6af868c1b6a43eea0c212a3c919bb7c P cbd8237e65464ef5883023baee5d6b66 [term 1 FOLLOWER]: Refusing update from remote peer f7f6545eb43942e88fcc745d92f1d0c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:07:45.756386 7389 raft_consensus.cc:1261] T c6af868c1b6a43eea0c212a3c919bb7c P e1bb28ca0e0f40a29165f730a8f48eb9 [term 1 FOLLOWER]: Refusing update from remote peer f7f6545eb43942e88fcc745d92f1d0c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:07:45.757385 7506 consensus_queue.cc:1035] T c6af868c1b6a43eea0c212a3c919bb7c P f7f6545eb43942e88fcc745d92f1d0c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1bb28ca0e0f40a29165f730a8f48eb9" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 33379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:07:45.757727 7499 consensus_queue.cc:1035] T c6af868c1b6a43eea0c212a3c919bb7c P f7f6545eb43942e88fcc745d92f1d0c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbd8237e65464ef5883023baee5d6b66" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 33709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:07:45.766958 7518 mvcc.cc:205] Tried to move back new op lower bound from 7317299469328093184 to 7317299469000052736. Current Snapshot: MvccSnapshot[applied={T|T < 7317299469328093184}]
I20260811 12:07:45.769268 7519 mvcc.cc:205] Tried to move back new op lower bound from 7317299469328093184 to 7317299469000052736. Current Snapshot: MvccSnapshot[applied={T|T < 7317299469328093184}]
I20260811 12:07:45.786648 6722 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:07:45.787029 6722 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:07:45.791874 6722 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 113251304764470b92d34432d9a97af4: Sending DeleteTablet for 3 replicas of tablet c6af868c1b6a43eea0c212a3c919bb7c
I20260811 12:07:45.793306 7362 tablet_service.cc:1514] Processing DeleteTablet for tablet c6af868c1b6a43eea0c212a3c919bb7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:07:45 UTC) from {username='slave'} at 127.0.0.1:44174
I20260811 12:07:45.793341 7147 tablet_service.cc:1514] Processing DeleteTablet for tablet c6af868c1b6a43eea0c212a3c919bb7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:07:45 UTC) from {username='slave'} at 127.0.0.1:47034
I20260811 12:07:45.793705 6931 tablet_service.cc:1514] Processing DeleteTablet for tablet c6af868c1b6a43eea0c212a3c919bb7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:07:45 UTC) from {username='slave'} at 127.0.0.1:39158
I20260811 12:07:45.793274 7361 tablet_service.cc:1514] Processing DeleteTablet for tablet c6af868c1b6a43eea0c212a3c919bb7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:07:45 UTC) from {username='slave'} at 127.0.0.1:44174
I20260811 12:07:45.796006 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 6825
I20260811 12:07:45.804570 7536 tablet_replica.cc:330] T c6af868c1b6a43eea0c212a3c919bb7c P e1bb28ca0e0f40a29165f730a8f48eb9: stopping tablet replica
I20260811 12:07:45.805225 7536 raft_consensus.cc:2229] T c6af868c1b6a43eea0c212a3c919bb7c P e1bb28ca0e0f40a29165f730a8f48eb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:07:45.805920 7536 raft_consensus.cc:2258] T c6af868c1b6a43eea0c212a3c919bb7c P e1bb28ca0e0f40a29165f730a8f48eb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:07:45.808440 7536 ts_tablet_manager.cc:1905] T c6af868c1b6a43eea0c212a3c919bb7c P e1bb28ca0e0f40a29165f730a8f48eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:07:45.811203 7537 tablet_replica.cc:330] T c6af868c1b6a43eea0c212a3c919bb7c P cbd8237e65464ef5883023baee5d6b66: stopping tablet replica
I20260811 12:07:45.811720 7537 raft_consensus.cc:2229] T c6af868c1b6a43eea0c212a3c919bb7c P cbd8237e65464ef5883023baee5d6b66 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:07:45.818636 7536 ts_tablet_manager.cc:1918] T c6af868c1b6a43eea0c212a3c919bb7c P e1bb28ca0e0f40a29165f730a8f48eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:07:45.818847 7536 log.cc:1196] T c6af868c1b6a43eea0c212a3c919bb7c P e1bb28ca0e0f40a29165f730a8f48eb9: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-2/wal/wals/c6af868c1b6a43eea0c212a3c919bb7c
I20260811 12:07:45.819533 7537 raft_consensus.cc:2258] T c6af868c1b6a43eea0c212a3c919bb7c P cbd8237e65464ef5883023baee5d6b66 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:07:45.822249 7537 ts_tablet_manager.cc:1905] T c6af868c1b6a43eea0c212a3c919bb7c P cbd8237e65464ef5883023baee5d6b66: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:07:45.824560 7538 tablet_replica.cc:330] T c6af868c1b6a43eea0c212a3c919bb7c P f7f6545eb43942e88fcc745d92f1d0c3: stopping tablet replica
I20260811 12:07:45.825019 7538 raft_consensus.cc:2229] T c6af868c1b6a43eea0c212a3c919bb7c P f7f6545eb43942e88fcc745d92f1d0c3 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:07:45.825740 7538 raft_consensus.cc:2258] T c6af868c1b6a43eea0c212a3c919bb7c P f7f6545eb43942e88fcc745d92f1d0c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:07:45.828181 7538 ts_tablet_manager.cc:1905] T c6af868c1b6a43eea0c212a3c919bb7c P f7f6545eb43942e88fcc745d92f1d0c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:07:45.828466 7536 ts_tablet_manager.cc:1939] T c6af868c1b6a43eea0c212a3c919bb7c P e1bb28ca0e0f40a29165f730a8f48eb9: Deleting consensus metadata
I20260811 12:07:45.829959 7537 ts_tablet_manager.cc:1918] T c6af868c1b6a43eea0c212a3c919bb7c P cbd8237e65464ef5883023baee5d6b66: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:07:45.830193 7537 log.cc:1196] T c6af868c1b6a43eea0c212a3c919bb7c P cbd8237e65464ef5883023baee5d6b66: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-1/wal/wals/c6af868c1b6a43eea0c212a3c919bb7c
I20260811 12:07:45.830561 6707 catalog_manager.cc:4841] TS e1bb28ca0e0f40a29165f730a8f48eb9 (127.9.221.131:33379): tablet c6af868c1b6a43eea0c212a3c919bb7c (table test-table [id=2e8c6fe10b25429ea8483e37b36ad602]) successfully deleted
I20260811 12:07:45.830884 7537 ts_tablet_manager.cc:1939] T c6af868c1b6a43eea0c212a3c919bb7c P cbd8237e65464ef5883023baee5d6b66: Deleting consensus metadata
W20260811 12:07:45.831697 6707 catalog_manager.cc:4808] TS e1bb28ca0e0f40a29165f730a8f48eb9 (127.9.221.131:33379): delete failed for tablet c6af868c1b6a43eea0c212a3c919bb7c because the tablet was not found. No further retry: Not found: Tablet not found: c6af868c1b6a43eea0c212a3c919bb7c
I20260811 12:07:45.832743 6703 catalog_manager.cc:4841] TS cbd8237e65464ef5883023baee5d6b66 (127.9.221.130:33709): tablet c6af868c1b6a43eea0c212a3c919bb7c (table test-table [id=2e8c6fe10b25429ea8483e37b36ad602]) successfully deleted
I20260811 12:07:45.834693 7538 ts_tablet_manager.cc:1918] T c6af868c1b6a43eea0c212a3c919bb7c P f7f6545eb43942e88fcc745d92f1d0c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:07:45.834890 7538 log.cc:1196] T c6af868c1b6a43eea0c212a3c919bb7c P f7f6545eb43942e88fcc745d92f1d0c3: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449743084027-10102-0/minicluster-data/ts-0/wal/wals/c6af868c1b6a43eea0c212a3c919bb7c
I20260811 12:07:45.835538 7538 ts_tablet_manager.cc:1939] T c6af868c1b6a43eea0c212a3c919bb7c P f7f6545eb43942e88fcc745d92f1d0c3: Deleting consensus metadata
I20260811 12:07:45.836984 6705 catalog_manager.cc:4841] TS f7f6545eb43942e88fcc745d92f1d0c3 (127.9.221.129:45201): tablet c6af868c1b6a43eea0c212a3c919bb7c (table test-table [id=2e8c6fe10b25429ea8483e37b36ad602]) successfully deleted
I20260811 12:07:45.837285 7485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:45.800605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47290 (local address 127.9.221.129:45201)
0811 12:07:45.800827 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:45.800836 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:45.819420 (+ 18584us) server_negotiation.cc:407] Connection header received
0811 12:07:45.819782 (+ 362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:45.819794 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:45.819905 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:45.819986 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:45.820733 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:45.822083 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:45.826127 (+ 4044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:45.826848 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:45.826955 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:45.835508 (+ 8553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:45.835519 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:45.835540 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:07:45.836347 (+ 807us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:45.836427 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:07:45.836453 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:45.836572 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:45.836792 (+ 220us) server_negotiation.cc:299] Negotiation successful
0811 12:07:45.836908 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 12:07:45.837975 7019 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:07:45.941689 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 6825
I20260811 12:07:45.966789 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 7038
I20260811 12:07:45.973654 7481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:45.967541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59124 (local address 127.9.221.130:33709)
0811 12:07:45.967761 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:45.967765 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:45.967784 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:07:45.967960 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:45.967965 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:45.968067 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:45.968151 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:45.968523 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:45.969279 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:45.970468 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:45.970865 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:45.970953 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:45.972029 (+ 1076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:45.972038 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:45.972054 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:07:45.972741 (+ 687us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:45.972802 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:07:45.972826 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:45.972929 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:45.973115 (+ 186us) server_negotiation.cc:299] Negotiation successful
0811 12:07:45.973213 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 12:07:45.974483 7249 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:07:46.064392 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 7038
I20260811 12:07:46.088502 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 7260
I20260811 12:07:46.095216 7480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:46.089370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33158 (local address 127.9.221.131:33379)
0811 12:07:46.089546 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:46.089550 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:46.089569 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:07:46.089708 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:46.089713 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:46.089799 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:46.089904 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:46.090277 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:46.091007 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:46.092388 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:46.092767 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:46.092883 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:46.093864 (+ 981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:46.093874 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:46.093891 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:07:46.094502 (+ 611us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:46.094552 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:07:46.094573 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:46.094673 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:46.094865 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 12:07:46.094960 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 12:07:46.095975 7452 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:07:46.184105 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 7260
I20260811 12:07:46.209071 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 6679
I20260811 12:07:46.210244 6779 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:07:46.319404 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 6679
2026-08-11T12:07:46Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3010 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:07:46 dist-test-slave-mjjr krb5kdc[7550](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:07:46 dist-test-slave-mjjr krb5kdc[7550](info): set up 2 sockets
Aug 11 12:07:46 dist-test-slave-mjjr krb5kdc[7550](info): commencing operation
krb5kdc: starting...
W20260811 12:07:47.719143 10102 mini_kdc.cc:121] Time spent starting KDC: real 1.348s user 0.000s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Aug 11 12:07:47 dist-test-slave-mjjr krb5kdc[7550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450067, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:07:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:07:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.221.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
I20260811 12:07:48.054526 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:33797
--webserver_interface=127.9.221.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:36193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:33797
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:07:48.394938 7575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:07:48.395208 7575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:07:48.395390 7575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:07:48.402509 7575 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:07:48.402606 7575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:07:48.402724 7575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:07:48.402827 7575 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:07:48.402910 7575 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:07:48.412602 7575 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:36193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:33797
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:33797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.221.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:07:48.413864 7575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:07:48.415164 7575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:07:48.427081 7592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:48.427134 7588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:48.427158 7587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:07:48.428301 7575 server_base.cc:1034] running on GCE node
I20260811 12:07:48.429033 7575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:07:48.430347 7575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:07:48.431578 7575 hybrid_clock.cc:648] HybridClock initialized: now 1786450068431531 us; error 56 us; skew 500 ppm
I20260811 12:07:48.431929 7575 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:07:48 dist-test-slave-mjjr krb5kdc[7550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450068, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:07:48.434927 7575 init.cc:377] Logged in from keytab as kudu/127.9.221.190@KRBTEST.COM (short username kudu)
I20260811 12:07:48.436338 7575 webserver.cc:460] Webserver started at http://127.9.221.190:45011/ using document root <none> and password file <none>
I20260811 12:07:48.436947 7575 fs_manager.cc:359] Metadata directory not provided
I20260811 12:07:48.437042 7575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:07:48.437331 7575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:07:48.440320 7575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "cf834652693146d093089526b285076b"
format_stamp: "Formatted at 2026-08-11 12:07:48 on dist-test-slave-mjjr"
server_key: "bf9305a6a887ae37b8624dedc9156353"
server_key_iv: "a10054716389edc8c22529e8bdc668c1"
server_key_version: "encryptionkey@0"
I20260811 12:07:48.441013 7575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "cf834652693146d093089526b285076b"
format_stamp: "Formatted at 2026-08-11 12:07:48 on dist-test-slave-mjjr"
server_key: "bf9305a6a887ae37b8624dedc9156353"
server_key_iv: "a10054716389edc8c22529e8bdc668c1"
server_key_version: "encryptionkey@0"
I20260811 12:07:48.445858 7575 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 12:07:48.449215 7606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:48.450569 7575 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 12:07:48.450713 7575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "cf834652693146d093089526b285076b"
format_stamp: "Formatted at 2026-08-11 12:07:48 on dist-test-slave-mjjr"
server_key: "bf9305a6a887ae37b8624dedc9156353"
server_key_iv: "a10054716389edc8c22529e8bdc668c1"
server_key_version: "encryptionkey@0"
I20260811 12:07:48.450870 7575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:07:48.466813 7575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:07:48.470746 7575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:07:48.470992 7575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:07:48.490087 7575 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.190:33797
I20260811 12:07:48.490113 7680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.190:33797 every 8 connection(s)
I20260811 12:07:48.491464 7575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:07:48.493170 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 7575
I20260811 12:07:48.493386 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:07:48.493829 10102 external_mini_cluster.cc:1428] Setting key 95b92f8c82ad841d924867c7e33f4979
I20260811 12:07:48.496214 7682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 12:07:48 dist-test-slave-mjjr krb5kdc[7550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450067, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:07:48.509439 7682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf834652693146d093089526b285076b: Bootstrap starting.
I20260811 12:07:48.513581 7682 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf834652693146d093089526b285076b: Neither blocks nor log segments found. Creating new log.
I20260811 12:07:48.514834 7682 log.cc:824] T 00000000000000000000000000000000 P cf834652693146d093089526b285076b: Log is configured to *not* fsync() on all Append() calls
I20260811 12:07:48.518275 7682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf834652693146d093089526b285076b: No bootstrap required, opened a new log
I20260811 12:07:48.518643 7691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:48.495754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50904 (local address 127.9.221.190:33797)
0811 12:07:48.496629 (+ 875us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:48.496645 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:48.496694 (+ 49us) server_negotiation.cc:407] Connection header received
0811 12:07:48.499120 (+ 2426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:48.499147 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:48.499676 (+ 529us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:48.500103 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:48.501475 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:48.503566 (+ 2091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:48.504846 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:48.505852 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:48.506037 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:48.510226 (+ 4189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:48.510235 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:48.510251 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:07:48.510297 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:48.514305 (+ 4008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:48.514986 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:48.515009 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:48.515018 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:48.515217 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:48.515689 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:48.515698 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:48.515700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:48.516221 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:48.516531 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:48.516857 (+ 326us) server_negotiation.cc:299] Negotiation successful
0811 12:07:48.517252 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":396,"threads_started":1}
I20260811 12:07:48.526226 7682 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf834652693146d093089526b285076b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf834652693146d093089526b285076b" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 33797 } }
I20260811 12:07:48.526674 7682 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf834652693146d093089526b285076b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:07:48.526758 7682 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf834652693146d093089526b285076b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf834652693146d093089526b285076b, State: Initialized, Role: FOLLOWER
I20260811 12:07:48.527379 7682 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf834652693146d093089526b285076b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf834652693146d093089526b285076b" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 33797 } }
I20260811 12:07:48.527612 7682 raft_consensus.cc:388] T 00000000000000000000000000000000 P cf834652693146d093089526b285076b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:07:48.527805 7682 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf834652693146d093089526b285076b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:07:48.527994 7682 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf834652693146d093089526b285076b [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:07:48.530936 7682 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf834652693146d093089526b285076b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf834652693146d093089526b285076b" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 33797 } }
I20260811 12:07:48.531523 7682 leader_election.cc:302] T 00000000000000000000000000000000 P cf834652693146d093089526b285076b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cf834652693146d093089526b285076b; no voters:
I20260811 12:07:48.532265 7682 leader_election.cc:288] T 00000000000000000000000000000000 P cf834652693146d093089526b285076b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:07:48.532434 7695 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf834652693146d093089526b285076b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:07:48.533146 7695 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf834652693146d093089526b285076b [term 1 LEADER]: Becoming Leader. State: Replica: cf834652693146d093089526b285076b, State: Running, Role: LEADER
I20260811 12:07:48.533686 7695 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf834652693146d093089526b285076b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf834652693146d093089526b285076b" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 33797 } }
I20260811 12:07:48.534330 7682 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf834652693146d093089526b285076b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:07:48.536837 7696 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf834652693146d093089526b285076b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cf834652693146d093089526b285076b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf834652693146d093089526b285076b" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 33797 } } }
I20260811 12:07:48.537271 7696 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf834652693146d093089526b285076b [sys.catalog]: This master's current role is: LEADER
I20260811 12:07:48.541816 7697 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf834652693146d093089526b285076b [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf834652693146d093089526b285076b. Latest consensus state: current_term: 1 leader_uuid: "cf834652693146d093089526b285076b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf834652693146d093089526b285076b" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 33797 } } }
I20260811 12:07:48.542784 7697 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf834652693146d093089526b285076b [sys.catalog]: This master's current role is: LEADER
I20260811 12:07:48.543574 7706 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:07:48.547832 7706 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:07:48.555792 7706 catalog_manager.cc:1347] Generated new cluster ID: ba6f560428c641b6b9c0a50faeffd85e
I20260811 12:07:48.555895 7706 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:07:48.573137 7706 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:07:48.575016 7706 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:07:48.586016 7706 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cf834652693146d093089526b285076b: Generated new TSK 0
I20260811 12:07:48.587093 7706 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.221.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab.
Entry for principal kudu/127.9.221.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab.
I20260811 12:07:48.653053 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.129:0
--local_ip_for_outbound_sockets=127.9.221.129
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:33797
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab
--principal=kudu/127.9.221.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:36193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:07:49.019418 7729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:07:49.019749 7729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:07:49.019912 7729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:07:49.027179 7729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:07:49.027311 7729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:07:49.027495 7729 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.129
I20260811 12:07:49.037271 7729 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:36193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.129.keytab
--principal=kudu/127.9.221.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:33797
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.129
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:07:49.038642 7729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:07:49.039987 7729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:07:49.053167 7741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:49.053244 7739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:49.053274 7745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:07:49.054441 7729 server_base.cc:1034] running on GCE node
I20260811 12:07:49.055052 7729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:07:49.056135 7729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:07:49.057340 7729 hybrid_clock.cc:648] HybridClock initialized: now 1786450069057298 us; error 37 us; skew 500 ppm
I20260811 12:07:49.057615 7729 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:07:49 dist-test-slave-mjjr krb5kdc[7550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450069, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:07:49.060796 7729 init.cc:377] Logged in from keytab as kudu/127.9.221.129@KRBTEST.COM (short username kudu)
I20260811 12:07:49.063063 7729 webserver.cc:460] Webserver started at http://127.9.221.129:37671/ using document root <none> and password file <none>
I20260811 12:07:49.063807 7729 fs_manager.cc:359] Metadata directory not provided
I20260811 12:07:49.063951 7729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:07:49.064301 7729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:07:49.067219 7729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-0/data/instance:
uuid: "73d0aeb063f343e9a8665162cf981f5e"
format_stamp: "Formatted at 2026-08-11 12:07:49 on dist-test-slave-mjjr"
server_key: "e3c2f235dd2ba70b40ad20e43ef651a8"
server_key_iv: "46544d46f713f90aad19bb48f291739b"
server_key_version: "encryptionkey@0"
I20260811 12:07:49.068044 7729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance:
uuid: "73d0aeb063f343e9a8665162cf981f5e"
format_stamp: "Formatted at 2026-08-11 12:07:49 on dist-test-slave-mjjr"
server_key: "e3c2f235dd2ba70b40ad20e43ef651a8"
server_key_iv: "46544d46f713f90aad19bb48f291739b"
server_key_version: "encryptionkey@0"
I20260811 12:07:49.073045 7729 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 12:07:49.076481 7758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:49.077757 7729 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:07:49.077941 7729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-0/wal
uuid: "73d0aeb063f343e9a8665162cf981f5e"
format_stamp: "Formatted at 2026-08-11 12:07:49 on dist-test-slave-mjjr"
server_key: "e3c2f235dd2ba70b40ad20e43ef651a8"
server_key_iv: "46544d46f713f90aad19bb48f291739b"
server_key_version: "encryptionkey@0"
I20260811 12:07:49.078125 7729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:07:49.097328 7729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:07:49.101753 7729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:07:49.102080 7729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:07:49.103091 7729 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:07:49.104494 7729 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:07:49.104569 7729 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:49.104743 7729 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:07:49.104815 7729 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:49.138723 7729 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.129:44051
I20260811 12:07:49.138842 7955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.129:44051 every 8 connection(s)
I20260811 12:07:49.140128 7729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
I20260811 12:07:49.145753 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 7729
I20260811 12:07:49.145992 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance
I20260811 12:07:49.146662 10102 external_mini_cluster.cc:1428] Setting key c9e8d81ff7018d216a870ace14dc7b82
Aug 11 12:07:49 dist-test-slave-mjjr krb5kdc[7550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450069, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.129@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:07:49.161278 7959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:49.142859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:54393 (local address 127.9.221.190:33797)
0811 12:07:49.143334 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:49.143340 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:49.144455 (+ 1115us) server_negotiation.cc:407] Connection header received
0811 12:07:49.146313 (+ 1858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:49.146321 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:49.146426 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:49.146580 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:49.148287 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:49.149159 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:49.150702 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:49.151123 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:49.151273 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:49.155143 (+ 3870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:49.155148 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:49.155152 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:07:49.155189 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:49.158156 (+ 2967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:49.158911 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:49.158917 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:49.158927 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:49.159047 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:49.159484 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:49.159488 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:49.159493 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:49.159824 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:49.159983 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:49.160684 (+ 701us) server_negotiation.cc:299] Negotiation successful
0811 12:07:49.160916 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":229,"threads_started":1}
I20260811 12:07:49.162233 7962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:49.143516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:33797 (local address 127.9.221.129:54393)
0811 12:07:49.144270 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:49.144326 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 12:07:49.145990 (+ 1664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:07:49.146813 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:07:49.146825 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:07:49.147474 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:07:49.148113 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:49.148130 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:49.149334 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:49.149338 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:49.150538 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:49.150550 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:49.151376 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:49.151384 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:49.151618 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:49.152460 (+ 842us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:07:49.152492 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:07:49.154906 (+ 2414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:07:49.158307 (+ 3401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:07:49.158316 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:07:49.158334 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:07:49.158768 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:07:49.159161 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:07:49.159167 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:07:49.159171 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:07:49.159361 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:07:49.159957 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:07:49.159965 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:07:49.160304 (+ 339us) client_negotiation.cc:769] Sending connection context
0811 12:07:49.160573 (+ 269us) client_negotiation.cc:240] Negotiation successful
0811 12:07:49.161114 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":331,"threads_started":1}
I20260811 12:07:49.164031 7956 heartbeater.cc:347] Connected to a master server at 127.9.221.190:33797
I20260811 12:07:49.164350 7956 heartbeater.cc:464] Registering TS with master...
I20260811 12:07:49.165062 7956 heartbeater.cc:511] Master 127.9.221.190:33797 requested a full tablet report, sending...
I20260811 12:07:49.167192 7634 ts_manager.cc:194] Registered new tserver with Master: 73d0aeb063f343e9a8665162cf981f5e (127.9.221.129:44051)
I20260811 12:07:49.168691 7634 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.129@KRBTEST.COM'} at 127.9.221.129:54393
WARNING: no policy specified for kudu/127.9.221.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab.
Entry for principal kudu/127.9.221.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab.
I20260811 12:07:49.214382 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.130:0
--local_ip_for_outbound_sockets=127.9.221.130
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:33797
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab
--principal=kudu/127.9.221.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:36193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:07:49.555192 7969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:07:49.555534 7969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:07:49.555734 7969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:07:49.562973 7969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:07:49.563073 7969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:07:49.563185 7969 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.130
I20260811 12:07:49.572871 7969 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:36193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.130.keytab
--principal=kudu/127.9.221.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:33797
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.130
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:07:49.574121 7969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:07:49.575409 7969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:07:49.589177 7982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:49.589221 7986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:49.589143 7983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:07:49.590334 7969 server_base.cc:1034] running on GCE node
I20260811 12:07:49.590888 7969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:07:49.591850 7969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:07:49.593083 7969 hybrid_clock.cc:648] HybridClock initialized: now 1786450069593045 us; error 27 us; skew 500 ppm
I20260811 12:07:49.593415 7969 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:07:49 dist-test-slave-mjjr krb5kdc[7550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450069, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:07:49.596623 7969 init.cc:377] Logged in from keytab as kudu/127.9.221.130@KRBTEST.COM (short username kudu)
I20260811 12:07:49.598026 7969 webserver.cc:460] Webserver started at http://127.9.221.130:43525/ using document root <none> and password file <none>
I20260811 12:07:49.598718 7969 fs_manager.cc:359] Metadata directory not provided
I20260811 12:07:49.598867 7969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:07:49.599190 7969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:07:49.602118 7969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-1/data/instance:
uuid: "987b9e22cd29460889b3fdf1501de118"
format_stamp: "Formatted at 2026-08-11 12:07:49 on dist-test-slave-mjjr"
server_key: "6c9638e2a819a534b8f6964a71ddc376"
server_key_iv: "9a53d47c5dff14e346b9ab51a33000dd"
server_key_version: "encryptionkey@0"
I20260811 12:07:49.602893 7969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance:
uuid: "987b9e22cd29460889b3fdf1501de118"
format_stamp: "Formatted at 2026-08-11 12:07:49 on dist-test-slave-mjjr"
server_key: "6c9638e2a819a534b8f6964a71ddc376"
server_key_iv: "9a53d47c5dff14e346b9ab51a33000dd"
server_key_version: "encryptionkey@0"
I20260811 12:07:49.608556 7969 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 12:07:49.611995 7997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:49.613265 7969 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 12:07:49.613443 7969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-1/wal
uuid: "987b9e22cd29460889b3fdf1501de118"
format_stamp: "Formatted at 2026-08-11 12:07:49 on dist-test-slave-mjjr"
server_key: "6c9638e2a819a534b8f6964a71ddc376"
server_key_iv: "9a53d47c5dff14e346b9ab51a33000dd"
server_key_version: "encryptionkey@0"
I20260811 12:07:49.613642 7969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:07:49.651795 7969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:07:49.656562 7969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:07:49.656875 7969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:07:49.657861 7969 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:07:49.659300 7969 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:07:49.659413 7969 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:49.659565 7969 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:07:49.659639 7969 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:49.692734 7969 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.130:40545
I20260811 12:07:49.692795 8181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.130:40545 every 8 connection(s)
I20260811 12:07:49.694108 7969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
I20260811 12:07:49.694747 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 7969
I20260811 12:07:49.694960 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance
I20260811 12:07:49.695595 10102 external_mini_cluster.cc:1428] Setting key 46bc12c882338f1e92dcbc605bf7e95c
Aug 11 12:07:49 dist-test-slave-mjjr krb5kdc[7550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450069, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.130@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:07:49.716118 8184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:49.696902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:47689 (local address 127.9.221.190:33797)
0811 12:07:49.697406 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:49.697412 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:49.698628 (+ 1216us) server_negotiation.cc:407] Connection header received
0811 12:07:49.700598 (+ 1970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:49.700606 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:49.700706 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:49.700835 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:49.702278 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:49.703465 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:49.705115 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:49.705868 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:49.706010 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:49.709781 (+ 3771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:49.709790 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:49.709798 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:07:49.709854 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:49.712712 (+ 2858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:49.713505 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:49.713513 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:49.713520 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:49.713688 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:49.714094 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:49.714101 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:49.714107 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:49.714495 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:49.714651 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:49.715287 (+ 636us) server_negotiation.cc:299] Negotiation successful
0811 12:07:49.715591 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":250,"threads_started":1}
I20260811 12:07:49.716843 8185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:49.697587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:33797 (local address 127.9.221.130:47689)
0811 12:07:49.698451 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:49.698498 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 12:07:49.700331 (+ 1833us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:07:49.701019 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:07:49.701026 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:07:49.701542 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:07:49.702102 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:49.702118 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:49.703631 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:49.703636 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:49.704961 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:49.704974 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:49.706088 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:49.706093 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:49.706301 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:49.707034 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:07:49.707058 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:07:49.709540 (+ 2482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:07:49.712873 (+ 3333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:07:49.712880 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:07:49.712890 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:07:49.713341 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:07:49.713798 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:07:49.713801 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:07:49.713803 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:07:49.713974 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:07:49.714614 (+ 640us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:07:49.714619 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:07:49.714917 (+ 298us) client_negotiation.cc:769] Sending connection context
0811 12:07:49.715128 (+ 211us) client_negotiation.cc:240] Negotiation successful
0811 12:07:49.715492 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":375,"threads_started":1}
I20260811 12:07:49.718524 8182 heartbeater.cc:347] Connected to a master server at 127.9.221.190:33797
I20260811 12:07:49.718822 8182 heartbeater.cc:464] Registering TS with master...
I20260811 12:07:49.719584 8182 heartbeater.cc:511] Master 127.9.221.190:33797 requested a full tablet report, sending...
I20260811 12:07:49.721060 7634 ts_manager.cc:194] Registered new tserver with Master: 987b9e22cd29460889b3fdf1501de118 (127.9.221.130:40545)
I20260811 12:07:49.721920 7634 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.130@KRBTEST.COM'} at 127.9.221.130:47689
WARNING: no policy specified for kudu/127.9.221.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab.
Entry for principal kudu/127.9.221.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab.
I20260811 12:07:49.765249 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.131:0
--local_ip_for_outbound_sockets=127.9.221.131
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:33797
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab
--principal=kudu/127.9.221.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:36193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:07:50.110369 8189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:07:50.110673 8189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:07:50.110841 8189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:07:50.117985 8189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:07:50.118160 8189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:07:50.118327 8189 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.131
I20260811 12:07:50.127707 8189 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:36193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.131.keytab
--principal=kudu/127.9.221.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:33797
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.131
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:07:50.128965 8189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:07:50.130264 8189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:07:50.147291 8197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:50.147346 8199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:50.147382 8203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:07:50.148241 8189 server_base.cc:1034] running on GCE node
I20260811 12:07:50.148855 8189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:07:50.149827 8189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:07:50.151063 8189 hybrid_clock.cc:648] HybridClock initialized: now 1786450070151000 us; error 55 us; skew 500 ppm
I20260811 12:07:50.151400 8189 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:07:50 dist-test-slave-mjjr krb5kdc[7550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450070, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:07:50.154796 8189 init.cc:377] Logged in from keytab as kudu/127.9.221.131@KRBTEST.COM (short username kudu)
I20260811 12:07:50.156431 8189 webserver.cc:460] Webserver started at http://127.9.221.131:44997/ using document root <none> and password file <none>
I20260811 12:07:50.157127 8189 fs_manager.cc:359] Metadata directory not provided
I20260811 12:07:50.157281 8189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:07:50.157605 8189 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:07:50.160666 8189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-2/data/instance:
uuid: "bb4acdd961e3443b8e51ac132f22843d"
format_stamp: "Formatted at 2026-08-11 12:07:50 on dist-test-slave-mjjr"
server_key: "cce75ae0993f8ea13e0bef92ac24b62a"
server_key_iv: "a91e3dc60923afd46536d9ccf0f3d8e4"
server_key_version: "encryptionkey@0"
I20260811 12:07:50.161449 8189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance:
uuid: "bb4acdd961e3443b8e51ac132f22843d"
format_stamp: "Formatted at 2026-08-11 12:07:50 on dist-test-slave-mjjr"
server_key: "cce75ae0993f8ea13e0bef92ac24b62a"
server_key_iv: "a91e3dc60923afd46536d9ccf0f3d8e4"
server_key_version: "encryptionkey@0"
I20260811 12:07:50.166671 8189 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 12:07:50.170055 8211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:50.171438 8189 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:07:50.171681 7956 heartbeater.cc:503] Master 127.9.221.190:33797 was elected leader, sending a full tablet report...
I20260811 12:07:50.171626 8189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-2/wal
uuid: "bb4acdd961e3443b8e51ac132f22843d"
format_stamp: "Formatted at 2026-08-11 12:07:50 on dist-test-slave-mjjr"
server_key: "cce75ae0993f8ea13e0bef92ac24b62a"
server_key_iv: "a91e3dc60923afd46536d9ccf0f3d8e4"
server_key_version: "encryptionkey@0"
I20260811 12:07:50.171877 8189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:07:50.207890 8189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:07:50.212301 8189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:07:50.212634 8189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:07:50.213632 8189 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:07:50.214967 8189 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:07:50.215083 8189 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:50.215235 8189 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:07:50.215317 8189 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:50.249801 8189 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.131:45059
I20260811 12:07:50.249891 8420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.131:45059 every 8 connection(s)
I20260811 12:07:50.251101 8189 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
I20260811 12:07:50.257131 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 8189
I20260811 12:07:50.257342 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance
I20260811 12:07:50.257897 10102 external_mini_cluster.cc:1428] Setting key e6cd70cab315a48b1421c5b8860e9c00
Aug 11 12:07:50 dist-test-slave-mjjr krb5kdc[7550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450070, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.131@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:07:50.271530 8423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:50.253661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:54133 (local address 127.9.221.190:33797)
0811 12:07:50.254153 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:50.254158 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:50.255055 (+ 897us) server_negotiation.cc:407] Connection header received
0811 12:07:50.257076 (+ 2021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:50.257080 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:50.257167 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:50.257310 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:50.259268 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:50.260289 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:50.261932 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:50.262349 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:50.262485 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:50.265840 (+ 3355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:50.265845 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:50.265848 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:07:50.265881 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:50.268422 (+ 2541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:50.269193 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:50.269202 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:50.269204 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:50.269322 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:50.269744 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:50.269750 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:50.269754 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:50.270024 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:50.270211 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:50.270901 (+ 690us) server_negotiation.cc:299] Negotiation successful
0811 12:07:50.271120 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":269,"threads_started":1}
I20260811 12:07:50.272568 8426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:50.254164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:33797 (local address 127.9.221.131:54133)
0811 12:07:50.254892 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:50.254930 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 12:07:50.256747 (+ 1817us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:07:50.257618 (+ 871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:07:50.257631 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:07:50.258392 (+ 761us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:07:50.259047 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:50.259075 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:50.260439 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:50.260444 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:50.261772 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:50.261783 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:50.262570 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:50.262578 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:50.262809 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:50.263438 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:07:50.263461 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:07:50.265630 (+ 2169us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:07:50.268565 (+ 2935us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:07:50.268574 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:07:50.268590 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:07:50.269068 (+ 478us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:07:50.269435 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:07:50.269440 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:07:50.269444 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:07:50.269629 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:07:50.270171 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:07:50.270178 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:07:50.270521 (+ 343us) client_negotiation.cc:769] Sending connection context
0811 12:07:50.270799 (+ 278us) client_negotiation.cc:240] Negotiation successful
0811 12:07:50.271322 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":304,"threads_started":1}
I20260811 12:07:50.273998 8421 heartbeater.cc:347] Connected to a master server at 127.9.221.190:33797
I20260811 12:07:50.274273 8421 heartbeater.cc:464] Registering TS with master...
I20260811 12:07:50.274994 8421 heartbeater.cc:511] Master 127.9.221.190:33797 requested a full tablet report, sending...
I20260811 12:07:50.276515 7634 ts_manager.cc:194] Registered new tserver with Master: bb4acdd961e3443b8e51ac132f22843d (127.9.221.131:45059)
I20260811 12:07:50.277269 7634 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.221.131@KRBTEST.COM'} at 127.9.221.131:54133
I20260811 12:07:50.285761 10102 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:07:50.301874 8423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:50.289908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50908 (local address 127.9.221.190:33797)
0811 12:07:50.290068 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:50.290073 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:50.290563 (+ 490us) server_negotiation.cc:407] Connection header received
0811 12:07:50.290696 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:50.290700 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:50.290761 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:50.290843 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:50.292319 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:50.293074 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:50.294283 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:50.294676 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:50.294770 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:50.297199 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:50.297205 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:50.297208 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:07:50.297239 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:50.299688 (+ 2449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:50.300275 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:50.300279 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:50.300281 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:50.300408 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:50.300713 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:50.300716 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:50.300718 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:50.301018 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:50.301198 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:50.301488 (+ 290us) server_negotiation.cc:299] Negotiation successful
0811 12:07:50.301650 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 12:07:50.305608 7634 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50908:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:07:50.312592 7634 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:07:50.347438 8450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:50.325101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60360 (local address 127.9.221.130:40545)
0811 12:07:50.339262 (+ 14161us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:50.339267 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:50.339289 (+ 22us) server_negotiation.cc:407] Connection header received
0811 12:07:50.339366 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:50.339370 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:50.339513 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:50.339722 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:07:50.340461 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:50.342096 (+ 1635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:50.344687 (+ 2591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:50.346205 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:50.346351 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:50.346586 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:50.346915 (+ 329us) server_negotiation.cc:299] Negotiation successful
0811 12:07:50.347101 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14017,"thread_start_us":13875,"threads_started":1}
I20260811 12:07:50.347499 8451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:50.324816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.130:40545 (local address 127.0.0.1:60360)
0811 12:07:50.338816 (+ 14000us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:50.338834 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:07:50.338949 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:07:50.339717 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:07:50.339721 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:07:50.339746 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:07:50.340257 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:50.340274 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:50.342244 (+ 1970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:50.342250 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:50.344466 (+ 2216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:50.344478 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:50.346409 (+ 1931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:50.346415 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:50.346623 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:50.346686 (+ 63us) client_negotiation.cc:769] Sending connection context
0811 12:07:50.346814 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 12:07:50.347001 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13892,"mutex_wait_us":80,"thread_start_us":11571,"threads_started":1}
I20260811 12:07:50.348424 8449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:50.324901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60522 (local address 127.9.221.129:44051)
0811 12:07:50.337899 (+ 12998us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:50.337909 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:50.337939 (+ 30us) server_negotiation.cc:407] Connection header received
0811 12:07:50.338033 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:50.338040 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:50.338347 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:50.338585 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:07:50.340008 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:50.341927 (+ 1919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:50.344661 (+ 2734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:50.346248 (+ 1587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:50.346381 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:50.346625 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:50.347917 (+ 1292us) server_negotiation.cc:299] Negotiation successful
0811 12:07:50.348095 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12826,"thread_start_us":12655,"threads_started":1}
I20260811 12:07:50.348791 8447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:50.324708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:44051 (local address 127.0.0.1:60522)
0811 12:07:50.336908 (+ 12200us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:50.336972 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 12:07:50.337204 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:07:50.339129 (+ 1925us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:07:50.339137 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:07:50.339175 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:07:50.339700 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:50.339717 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:50.342091 (+ 2374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:50.342096 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:50.344364 (+ 2268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:50.344431 (+ 67us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:50.346753 (+ 2322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:50.346762 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:50.346936 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:50.346967 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 12:07:50.347050 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 12:07:50.347177 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11837,"mutex_wait_us":110,"thread_start_us":15166,"threads_started":1}
I20260811 12:07:50.348865 8444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:50.325878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.131:45059 (local address 127.0.0.1:50050)
0811 12:07:50.339630 (+ 13752us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:50.339673 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 12:07:50.339799 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:07:50.340433 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:07:50.340437 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:07:50.340459 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:07:50.340829 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:50.340838 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:50.343545 (+ 2707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:50.343551 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:50.345942 (+ 2391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:50.345959 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:50.348025 (+ 2066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:50.348033 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:50.348198 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:50.348230 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 12:07:50.348342 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 12:07:50.348477 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13593,"mutex_wait_us":1,"thread_start_us":13104,"threads_started":1}
I20260811 12:07:50.349968 8452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:50.326602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50050 (local address 127.9.221.131:45059)
0811 12:07:50.338421 (+ 11819us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:50.338430 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:50.339943 (+ 1513us) server_negotiation.cc:407] Connection header received
0811 12:07:50.340043 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:50.340051 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:50.340225 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:50.340426 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:07:50.341220 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:50.343388 (+ 2168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:50.346121 (+ 2733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:50.347544 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:50.347709 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:50.347936 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:50.348410 (+ 474us) server_negotiation.cc:299] Negotiation successful
0811 12:07:50.348590 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11644,"thread_start_us":11461,"threads_started":1}
I20260811 12:07:50.351908 8075 tablet_service.cc:1467] Processing CreateTablet for tablet 5972ef6fbe684d5c92943d3bcb493fd3 (DEFAULT_TABLE table=test-table [id=442dceb1de2940bc9f4c131e49d42184]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:07:50.352331 7847 tablet_service.cc:1467] Processing CreateTablet for tablet 5972ef6fbe684d5c92943d3bcb493fd3 (DEFAULT_TABLE table=test-table [id=442dceb1de2940bc9f4c131e49d42184]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:07:50.352982 8302 tablet_service.cc:1467] Processing CreateTablet for tablet 5972ef6fbe684d5c92943d3bcb493fd3 (DEFAULT_TABLE table=test-table [id=442dceb1de2940bc9f4c131e49d42184]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:07:50.353454 8075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5972ef6fbe684d5c92943d3bcb493fd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:07:50.353652 7847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5972ef6fbe684d5c92943d3bcb493fd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:07:50.354197 8302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5972ef6fbe684d5c92943d3bcb493fd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:07:50.365285 8456 tablet_bootstrap.cc:492] T 5972ef6fbe684d5c92943d3bcb493fd3 P bb4acdd961e3443b8e51ac132f22843d: Bootstrap starting.
I20260811 12:07:50.368173 8457 tablet_bootstrap.cc:492] T 5972ef6fbe684d5c92943d3bcb493fd3 P 987b9e22cd29460889b3fdf1501de118: Bootstrap starting.
I20260811 12:07:50.369611 8459 tablet_bootstrap.cc:492] T 5972ef6fbe684d5c92943d3bcb493fd3 P 73d0aeb063f343e9a8665162cf981f5e: Bootstrap starting.
I20260811 12:07:50.370018 8456 tablet_bootstrap.cc:654] T 5972ef6fbe684d5c92943d3bcb493fd3 P bb4acdd961e3443b8e51ac132f22843d: Neither blocks nor log segments found. Creating new log.
I20260811 12:07:50.371415 8456 log.cc:824] T 5972ef6fbe684d5c92943d3bcb493fd3 P bb4acdd961e3443b8e51ac132f22843d: Log is configured to *not* fsync() on all Append() calls
I20260811 12:07:50.371815 8457 tablet_bootstrap.cc:654] T 5972ef6fbe684d5c92943d3bcb493fd3 P 987b9e22cd29460889b3fdf1501de118: Neither blocks nor log segments found. Creating new log.
I20260811 12:07:50.372612 8459 tablet_bootstrap.cc:654] T 5972ef6fbe684d5c92943d3bcb493fd3 P 73d0aeb063f343e9a8665162cf981f5e: Neither blocks nor log segments found. Creating new log.
I20260811 12:07:50.373356 8457 log.cc:824] T 5972ef6fbe684d5c92943d3bcb493fd3 P 987b9e22cd29460889b3fdf1501de118: Log is configured to *not* fsync() on all Append() calls
I20260811 12:07:50.373992 8459 log.cc:824] T 5972ef6fbe684d5c92943d3bcb493fd3 P 73d0aeb063f343e9a8665162cf981f5e: Log is configured to *not* fsync() on all Append() calls
I20260811 12:07:50.377511 8459 tablet_bootstrap.cc:492] T 5972ef6fbe684d5c92943d3bcb493fd3 P 73d0aeb063f343e9a8665162cf981f5e: No bootstrap required, opened a new log
I20260811 12:07:50.377799 8459 ts_tablet_manager.cc:1397] T 5972ef6fbe684d5c92943d3bcb493fd3 P 73d0aeb063f343e9a8665162cf981f5e: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260811 12:07:50.377853 8456 tablet_bootstrap.cc:492] T 5972ef6fbe684d5c92943d3bcb493fd3 P bb4acdd961e3443b8e51ac132f22843d: No bootstrap required, opened a new log
I20260811 12:07:50.377844 8457 tablet_bootstrap.cc:492] T 5972ef6fbe684d5c92943d3bcb493fd3 P 987b9e22cd29460889b3fdf1501de118: No bootstrap required, opened a new log
I20260811 12:07:50.378099 8456 ts_tablet_manager.cc:1397] T 5972ef6fbe684d5c92943d3bcb493fd3 P bb4acdd961e3443b8e51ac132f22843d: Time spent bootstrapping tablet: real 0.013s user 0.010s sys 0.000s
I20260811 12:07:50.378098 8457 ts_tablet_manager.cc:1397] T 5972ef6fbe684d5c92943d3bcb493fd3 P 987b9e22cd29460889b3fdf1501de118: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.008s
I20260811 12:07:50.385895 8459 raft_consensus.cc:348] T 5972ef6fbe684d5c92943d3bcb493fd3 P 73d0aeb063f343e9a8665162cf981f5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73d0aeb063f343e9a8665162cf981f5e" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 44051 } } peers { permanent_uuid: "987b9e22cd29460889b3fdf1501de118" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 40545 } } peers { permanent_uuid: "bb4acdd961e3443b8e51ac132f22843d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 45059 } }
I20260811 12:07:50.386277 8459 raft_consensus.cc:374] T 5972ef6fbe684d5c92943d3bcb493fd3 P 73d0aeb063f343e9a8665162cf981f5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:07:50.386404 8459 raft_consensus.cc:729] T 5972ef6fbe684d5c92943d3bcb493fd3 P 73d0aeb063f343e9a8665162cf981f5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73d0aeb063f343e9a8665162cf981f5e, State: Initialized, Role: FOLLOWER
I20260811 12:07:50.387121 8459 consensus_queue.cc:260] T 5972ef6fbe684d5c92943d3bcb493fd3 P 73d0aeb063f343e9a8665162cf981f5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73d0aeb063f343e9a8665162cf981f5e" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 44051 } } peers { permanent_uuid: "987b9e22cd29460889b3fdf1501de118" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 40545 } } peers { permanent_uuid: "bb4acdd961e3443b8e51ac132f22843d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 45059 } }
I20260811 12:07:50.389045 8457 raft_consensus.cc:348] T 5972ef6fbe684d5c92943d3bcb493fd3 P 987b9e22cd29460889b3fdf1501de118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73d0aeb063f343e9a8665162cf981f5e" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 44051 } } peers { permanent_uuid: "987b9e22cd29460889b3fdf1501de118" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 40545 } } peers { permanent_uuid: "bb4acdd961e3443b8e51ac132f22843d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 45059 } }
I20260811 12:07:50.389045 8456 raft_consensus.cc:348] T 5972ef6fbe684d5c92943d3bcb493fd3 P bb4acdd961e3443b8e51ac132f22843d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73d0aeb063f343e9a8665162cf981f5e" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 44051 } } peers { permanent_uuid: "987b9e22cd29460889b3fdf1501de118" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 40545 } } peers { permanent_uuid: "bb4acdd961e3443b8e51ac132f22843d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 45059 } }
I20260811 12:07:50.389467 8457 raft_consensus.cc:374] T 5972ef6fbe684d5c92943d3bcb493fd3 P 987b9e22cd29460889b3fdf1501de118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:07:50.389467 8456 raft_consensus.cc:374] T 5972ef6fbe684d5c92943d3bcb493fd3 P bb4acdd961e3443b8e51ac132f22843d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:07:50.389561 8457 raft_consensus.cc:729] T 5972ef6fbe684d5c92943d3bcb493fd3 P 987b9e22cd29460889b3fdf1501de118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 987b9e22cd29460889b3fdf1501de118, State: Initialized, Role: FOLLOWER
I20260811 12:07:50.389561 8456 raft_consensus.cc:729] T 5972ef6fbe684d5c92943d3bcb493fd3 P bb4acdd961e3443b8e51ac132f22843d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb4acdd961e3443b8e51ac132f22843d, State: Initialized, Role: FOLLOWER
I20260811 12:07:50.390125 8459 ts_tablet_manager.cc:1428] T 5972ef6fbe684d5c92943d3bcb493fd3 P 73d0aeb063f343e9a8665162cf981f5e: Time spent starting tablet: real 0.012s user 0.011s sys 0.001s
I20260811 12:07:50.390254 8457 consensus_queue.cc:260] T 5972ef6fbe684d5c92943d3bcb493fd3 P 987b9e22cd29460889b3fdf1501de118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73d0aeb063f343e9a8665162cf981f5e" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 44051 } } peers { permanent_uuid: "987b9e22cd29460889b3fdf1501de118" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 40545 } } peers { permanent_uuid: "bb4acdd961e3443b8e51ac132f22843d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 45059 } }
I20260811 12:07:50.390254 8456 consensus_queue.cc:260] T 5972ef6fbe684d5c92943d3bcb493fd3 P bb4acdd961e3443b8e51ac132f22843d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73d0aeb063f343e9a8665162cf981f5e" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 44051 } } peers { permanent_uuid: "987b9e22cd29460889b3fdf1501de118" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 40545 } } peers { permanent_uuid: "bb4acdd961e3443b8e51ac132f22843d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 45059 } }
I20260811 12:07:50.391901 8421 heartbeater.cc:503] Master 127.9.221.190:33797 was elected leader, sending a full tablet report...
I20260811 12:07:50.392066 8457 ts_tablet_manager.cc:1428] T 5972ef6fbe684d5c92943d3bcb493fd3 P 987b9e22cd29460889b3fdf1501de118: Time spent starting tablet: real 0.014s user 0.014s sys 0.001s
W20260811 12:07:50.392539 7957 tablet.cc:2369] T 5972ef6fbe684d5c92943d3bcb493fd3 P 73d0aeb063f343e9a8665162cf981f5e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:07:50.393440 8456 ts_tablet_manager.cc:1428] T 5972ef6fbe684d5c92943d3bcb493fd3 P bb4acdd961e3443b8e51ac132f22843d: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260811 12:07:50.395546 8182 heartbeater.cc:503] Master 127.9.221.190:33797 was elected leader, sending a full tablet report...
W20260811 12:07:50.446612 8183 tablet.cc:2369] T 5972ef6fbe684d5c92943d3bcb493fd3 P 987b9e22cd29460889b3fdf1501de118: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:07:50.497175 8469 raft_consensus.cc:482] T 5972ef6fbe684d5c92943d3bcb493fd3 P bb4acdd961e3443b8e51ac132f22843d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:07:50.497519 8469 raft_consensus.cc:504] T 5972ef6fbe684d5c92943d3bcb493fd3 P bb4acdd961e3443b8e51ac132f22843d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73d0aeb063f343e9a8665162cf981f5e" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 44051 } } peers { permanent_uuid: "987b9e22cd29460889b3fdf1501de118" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 40545 } } peers { permanent_uuid: "bb4acdd961e3443b8e51ac132f22843d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 45059 } }
I20260811 12:07:50.499255 8469 leader_election.cc:288] T 5972ef6fbe684d5c92943d3bcb493fd3 P bb4acdd961e3443b8e51ac132f22843d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73d0aeb063f343e9a8665162cf981f5e (127.9.221.129:44051), 987b9e22cd29460889b3fdf1501de118 (127.9.221.130:40545)
W20260811 12:07:50.502826 8422 tablet.cc:2369] T 5972ef6fbe684d5c92943d3bcb493fd3 P bb4acdd961e3443b8e51ac132f22843d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:07:50.505846 8450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:50.499741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:45215 (local address 127.9.221.130:40545)
0811 12:07:50.499972 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:50.499978 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:50.499999 (+ 21us) server_negotiation.cc:407] Connection header received
0811 12:07:50.500232 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:50.500238 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:50.500319 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:50.500406 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:07:50.500992 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:50.501931 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:50.504165 (+ 2234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:50.504955 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:50.505160 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:50.505236 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:50.505485 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 12:07:50.505609 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 12:07:50.505851 8426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:50.499529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.130:40545 (local address 127.9.221.131:45215)
0811 12:07:50.499857 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:50.499879 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:07:50.500020 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:07:50.500603 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:07:50.500608 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:07:50.500637 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:07:50.500842 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:50.500851 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:50.502061 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:50.502067 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:50.504011 (+ 1944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:50.504026 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:50.505132 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:50.505138 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:50.505320 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:50.505344 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 12:07:50.505447 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 12:07:50.505576 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":132}
I20260811 12:07:50.507050 8472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:50.499584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:44051 (local address 127.9.221.131:60323)
0811 12:07:50.500475 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:50.500492 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:07:50.500660 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:07:50.501021 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:07:50.501025 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:07:50.501055 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:07:50.501411 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:50.501423 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:50.502846 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:50.502852 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:50.505053 (+ 2201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:50.505068 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:50.506491 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:50.506495 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:50.506607 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:50.506629 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 12:07:50.506702 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 12:07:50.506781 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":753,"mutex_wait_us":2,"thread_start_us":775,"threads_started":1}
I20260811 12:07:50.507166 8449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:50.500194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:60323 (local address 127.9.221.129:44051)
0811 12:07:50.500391 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:50.500398 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:50.500574 (+ 176us) server_negotiation.cc:407] Connection header received
0811 12:07:50.500829 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:50.500834 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:50.500908 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:50.500996 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:07:50.501585 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:50.502644 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:50.505229 (+ 2585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:50.506289 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:50.506447 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:50.506544 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:50.506767 (+ 223us) server_negotiation.cc:299] Negotiation successful
0811 12:07:50.506901 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 12:07:50.506989 8107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5972ef6fbe684d5c92943d3bcb493fd3" candidate_uuid: "bb4acdd961e3443b8e51ac132f22843d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "987b9e22cd29460889b3fdf1501de118" is_pre_election: true
I20260811 12:07:50.507614 8107 raft_consensus.cc:2454] T 5972ef6fbe684d5c92943d3bcb493fd3 P 987b9e22cd29460889b3fdf1501de118 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb4acdd961e3443b8e51ac132f22843d in term 0.
I20260811 12:07:50.508296 7879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5972ef6fbe684d5c92943d3bcb493fd3" candidate_uuid: "bb4acdd961e3443b8e51ac132f22843d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73d0aeb063f343e9a8665162cf981f5e" is_pre_election: true
I20260811 12:07:50.508425 8217 leader_election.cc:302] T 5972ef6fbe684d5c92943d3bcb493fd3 P bb4acdd961e3443b8e51ac132f22843d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 987b9e22cd29460889b3fdf1501de118, bb4acdd961e3443b8e51ac132f22843d; no voters:
I20260811 12:07:50.508910 8469 raft_consensus.cc:2781] T 5972ef6fbe684d5c92943d3bcb493fd3 P bb4acdd961e3443b8e51ac132f22843d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:07:50.508889 7879 raft_consensus.cc:2454] T 5972ef6fbe684d5c92943d3bcb493fd3 P 73d0aeb063f343e9a8665162cf981f5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb4acdd961e3443b8e51ac132f22843d in term 0.
I20260811 12:07:50.509074 8469 raft_consensus.cc:482] T 5972ef6fbe684d5c92943d3bcb493fd3 P bb4acdd961e3443b8e51ac132f22843d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:07:50.509207 8469 raft_consensus.cc:3037] T 5972ef6fbe684d5c92943d3bcb493fd3 P bb4acdd961e3443b8e51ac132f22843d [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:07:50.511754 8469 raft_consensus.cc:504] T 5972ef6fbe684d5c92943d3bcb493fd3 P bb4acdd961e3443b8e51ac132f22843d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73d0aeb063f343e9a8665162cf981f5e" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 44051 } } peers { permanent_uuid: "987b9e22cd29460889b3fdf1501de118" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 40545 } } peers { permanent_uuid: "bb4acdd961e3443b8e51ac132f22843d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 45059 } }
I20260811 12:07:50.512478 8469 leader_election.cc:288] T 5972ef6fbe684d5c92943d3bcb493fd3 P bb4acdd961e3443b8e51ac132f22843d [CANDIDATE]: Term 1 election: Requested vote from peers 73d0aeb063f343e9a8665162cf981f5e (127.9.221.129:44051), 987b9e22cd29460889b3fdf1501de118 (127.9.221.130:40545)
I20260811 12:07:50.512887 7879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5972ef6fbe684d5c92943d3bcb493fd3" candidate_uuid: "bb4acdd961e3443b8e51ac132f22843d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73d0aeb063f343e9a8665162cf981f5e"
I20260811 12:07:50.512961 8107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5972ef6fbe684d5c92943d3bcb493fd3" candidate_uuid: "bb4acdd961e3443b8e51ac132f22843d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "987b9e22cd29460889b3fdf1501de118"
I20260811 12:07:50.513120 7879 raft_consensus.cc:3037] T 5972ef6fbe684d5c92943d3bcb493fd3 P 73d0aeb063f343e9a8665162cf981f5e [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:07:50.513175 8107 raft_consensus.cc:3037] T 5972ef6fbe684d5c92943d3bcb493fd3 P 987b9e22cd29460889b3fdf1501de118 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:07:50.516707 7879 raft_consensus.cc:2454] T 5972ef6fbe684d5c92943d3bcb493fd3 P 73d0aeb063f343e9a8665162cf981f5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb4acdd961e3443b8e51ac132f22843d in term 1.
I20260811 12:07:50.516705 8107 raft_consensus.cc:2454] T 5972ef6fbe684d5c92943d3bcb493fd3 P 987b9e22cd29460889b3fdf1501de118 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb4acdd961e3443b8e51ac132f22843d in term 1.
I20260811 12:07:50.517179 8217 leader_election.cc:302] T 5972ef6fbe684d5c92943d3bcb493fd3 P bb4acdd961e3443b8e51ac132f22843d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 987b9e22cd29460889b3fdf1501de118, bb4acdd961e3443b8e51ac132f22843d; no voters:
I20260811 12:07:50.517509 8469 raft_consensus.cc:2781] T 5972ef6fbe684d5c92943d3bcb493fd3 P bb4acdd961e3443b8e51ac132f22843d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:07:50.518008 8469 raft_consensus.cc:686] T 5972ef6fbe684d5c92943d3bcb493fd3 P bb4acdd961e3443b8e51ac132f22843d [term 1 LEADER]: Becoming Leader. State: Replica: bb4acdd961e3443b8e51ac132f22843d, State: Running, Role: LEADER
I20260811 12:07:50.518591 8469 consensus_queue.cc:237] T 5972ef6fbe684d5c92943d3bcb493fd3 P bb4acdd961e3443b8e51ac132f22843d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73d0aeb063f343e9a8665162cf981f5e" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 44051 } } peers { permanent_uuid: "987b9e22cd29460889b3fdf1501de118" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 40545 } } peers { permanent_uuid: "bb4acdd961e3443b8e51ac132f22843d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 45059 } }
I20260811 12:07:50.523154 7633 catalog_manager.cc:5495] T 5972ef6fbe684d5c92943d3bcb493fd3 P bb4acdd961e3443b8e51ac132f22843d reported cstate change: term changed from 0 to 1, leader changed from <none> to bb4acdd961e3443b8e51ac132f22843d (127.9.221.131). New cstate: current_term: 1 leader_uuid: "bb4acdd961e3443b8e51ac132f22843d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73d0aeb063f343e9a8665162cf981f5e" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 44051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "987b9e22cd29460889b3fdf1501de118" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 40545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb4acdd961e3443b8e51ac132f22843d" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 45059 } health_report { overall_health: HEALTHY } } }
I20260811 12:07:50.546319 8452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:50.540927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50066 (local address 127.9.221.131:45059)
0811 12:07:50.541228 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:50.541232 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:50.541251 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:07:50.541404 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:50.541408 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:50.541494 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:50.541620 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:07:50.541950 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:50.542656 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:50.544160 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:50.544553 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:50.544651 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:50.545157 (+ 506us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:07:50.545706 (+ 549us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:07:50.545855 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:50.545906 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 12:07:50.545979 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185}
I20260811 12:07:50.554713 7879 raft_consensus.cc:1261] T 5972ef6fbe684d5c92943d3bcb493fd3 P 73d0aeb063f343e9a8665162cf981f5e [term 1 FOLLOWER]: Refusing update from remote peer bb4acdd961e3443b8e51ac132f22843d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:07:50.554715 8107 raft_consensus.cc:1261] T 5972ef6fbe684d5c92943d3bcb493fd3 P 987b9e22cd29460889b3fdf1501de118 [term 1 FOLLOWER]: Refusing update from remote peer bb4acdd961e3443b8e51ac132f22843d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:07:50.555774 8469 consensus_queue.cc:1035] T 5972ef6fbe684d5c92943d3bcb493fd3 P bb4acdd961e3443b8e51ac132f22843d [LEADER]: Connected to new peer: Peer: permanent_uuid: "73d0aeb063f343e9a8665162cf981f5e" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 44051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:07:50.556087 8473 consensus_queue.cc:1035] T 5972ef6fbe684d5c92943d3bcb493fd3 P bb4acdd961e3443b8e51ac132f22843d [LEADER]: Connected to new peer: Peer: permanent_uuid: "987b9e22cd29460889b3fdf1501de118" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:07:50.568071 8477 mvcc.cc:205] Tried to move back new op lower bound from 7317299488981266432 to 7317299488848011264. Current Snapshot: MvccSnapshot[applied={T|T < 7317299488981266432}]
I20260811 12:07:50.581072 7634 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50908:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:07:50.581415 7634 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50908:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:07:50.585287 7634 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cf834652693146d093089526b285076b: Sending DeleteTablet for 3 replicas of tablet 5972ef6fbe684d5c92943d3bcb493fd3
I20260811 12:07:50.586536 7847 tablet_service.cc:1514] Processing DeleteTablet for tablet 5972ef6fbe684d5c92943d3bcb493fd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:07:50 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:60522
I20260811 12:07:50.586607 8302 tablet_service.cc:1514] Processing DeleteTablet for tablet 5972ef6fbe684d5c92943d3bcb493fd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:07:50 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:50050
I20260811 12:07:50.587402 8075 tablet_service.cc:1514] Processing DeleteTablet for tablet 5972ef6fbe684d5c92943d3bcb493fd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:07:50 UTC) from {username='kudu', principal='kudu/127.9.221.190@KRBTEST.COM'} at 127.0.0.1:60360
I20260811 12:07:50.588068 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 7729
I20260811 12:07:50.599725 8500 tablet_replica.cc:330] T 5972ef6fbe684d5c92943d3bcb493fd3 P 987b9e22cd29460889b3fdf1501de118: stopping tablet replica
I20260811 12:07:50.600236 8500 raft_consensus.cc:2229] T 5972ef6fbe684d5c92943d3bcb493fd3 P 987b9e22cd29460889b3fdf1501de118 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:07:50.600800 8500 raft_consensus.cc:2258] T 5972ef6fbe684d5c92943d3bcb493fd3 P 987b9e22cd29460889b3fdf1501de118 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:07:50.603196 8500 ts_tablet_manager.cc:1905] T 5972ef6fbe684d5c92943d3bcb493fd3 P 987b9e22cd29460889b3fdf1501de118: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:07:50.603812 8498 tablet_replica.cc:330] T 5972ef6fbe684d5c92943d3bcb493fd3 P bb4acdd961e3443b8e51ac132f22843d: stopping tablet replica
I20260811 12:07:50.604306 8498 raft_consensus.cc:2229] T 5972ef6fbe684d5c92943d3bcb493fd3 P bb4acdd961e3443b8e51ac132f22843d [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:07:50.605139 8498 raft_consensus.cc:2258] T 5972ef6fbe684d5c92943d3bcb493fd3 P bb4acdd961e3443b8e51ac132f22843d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:07:50.608032 8499 tablet_replica.cc:330] T 5972ef6fbe684d5c92943d3bcb493fd3 P 73d0aeb063f343e9a8665162cf981f5e: stopping tablet replica
I20260811 12:07:50.608495 8499 raft_consensus.cc:2229] T 5972ef6fbe684d5c92943d3bcb493fd3 P 73d0aeb063f343e9a8665162cf981f5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:07:50.609019 8499 raft_consensus.cc:2258] T 5972ef6fbe684d5c92943d3bcb493fd3 P 73d0aeb063f343e9a8665162cf981f5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:07:50.611238 8499 ts_tablet_manager.cc:1905] T 5972ef6fbe684d5c92943d3bcb493fd3 P 73d0aeb063f343e9a8665162cf981f5e: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 12:07:50 dist-test-slave-mjjr krb5kdc[7550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450067, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.129@KRBTEST.COM
I20260811 12:07:50.613337 8498 ts_tablet_manager.cc:1905] T 5972ef6fbe684d5c92943d3bcb493fd3 P bb4acdd961e3443b8e51ac132f22843d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:07:50.618959 8500 ts_tablet_manager.cc:1918] T 5972ef6fbe684d5c92943d3bcb493fd3 P 987b9e22cd29460889b3fdf1501de118: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:07:50.619143 8500 log.cc:1196] T 5972ef6fbe684d5c92943d3bcb493fd3 P 987b9e22cd29460889b3fdf1501de118: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-1/wal/wals/5972ef6fbe684d5c92943d3bcb493fd3
I20260811 12:07:50.621217 8498 ts_tablet_manager.cc:1918] T 5972ef6fbe684d5c92943d3bcb493fd3 P bb4acdd961e3443b8e51ac132f22843d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:07:50.621472 8498 log.cc:1196] T 5972ef6fbe684d5c92943d3bcb493fd3 P bb4acdd961e3443b8e51ac132f22843d: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-2/wal/wals/5972ef6fbe684d5c92943d3bcb493fd3
I20260811 12:07:50.621676 8449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:50.589368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60534 (local address 127.9.221.129:44051)
0811 12:07:50.589619 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:50.589628 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:50.598700 (+ 9072us) server_negotiation.cc:407] Connection header received
0811 12:07:50.598938 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:50.598946 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:50.599059 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:50.599194 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:50.603698 (+ 4504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:50.605344 (+ 1646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:50.607934 (+ 2590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:50.608463 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:50.608622 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:50.613692 (+ 5070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:50.613705 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:50.613721 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:07:50.613763 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:50.617979 (+ 4216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:50.618870 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:50.618881 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:50.618884 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:50.619052 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:50.619535 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:50.619540 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:50.619543 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:50.620021 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:50.620334 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:50.620936 (+ 602us) server_negotiation.cc:299] Negotiation successful
0811 12:07:50.621221 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 12:07:50.622434 8498 ts_tablet_manager.cc:1939] T 5972ef6fbe684d5c92943d3bcb493fd3 P bb4acdd961e3443b8e51ac132f22843d: Deleting consensus metadata
I20260811 12:07:50.622920 8499 ts_tablet_manager.cc:1918] T 5972ef6fbe684d5c92943d3bcb493fd3 P 73d0aeb063f343e9a8665162cf981f5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:07:50.623088 8499 log.cc:1196] T 5972ef6fbe684d5c92943d3bcb493fd3 P 73d0aeb063f343e9a8665162cf981f5e: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449743084027-10102-0/minicluster-data/ts-0/wal/wals/5972ef6fbe684d5c92943d3bcb493fd3
I20260811 12:07:50.623255 7947 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:07:50.623761 8500 ts_tablet_manager.cc:1939] T 5972ef6fbe684d5c92943d3bcb493fd3 P 987b9e22cd29460889b3fdf1501de118: Deleting consensus metadata
I20260811 12:07:50.624982 7610 catalog_manager.cc:4841] TS bb4acdd961e3443b8e51ac132f22843d (127.9.221.131:45059): tablet 5972ef6fbe684d5c92943d3bcb493fd3 (table test-table [id=442dceb1de2940bc9f4c131e49d42184]) successfully deleted
I20260811 12:07:50.625622 7611 catalog_manager.cc:4841] TS 987b9e22cd29460889b3fdf1501de118 (127.9.221.130:40545): tablet 5972ef6fbe684d5c92943d3bcb493fd3 (table test-table [id=442dceb1de2940bc9f4c131e49d42184]) successfully deleted
I20260811 12:07:50.729991 8499 ts_tablet_manager.cc:1939] T 5972ef6fbe684d5c92943d3bcb493fd3 P 73d0aeb063f343e9a8665162cf981f5e: Deleting consensus metadata
I20260811 12:07:50.730652 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 7729
I20260811 12:07:50.761289 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 7969
W20260811 12:07:50.761253 7608 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.221.129:44051 (error 108)
I20260811 12:07:50.763588 8447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:50.763039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:44051 (local address 127.0.0.1:60540)
0811 12:07:50.763260 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:50.763399 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.129:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260811 12:07:50.763895 7608 catalog_manager.cc:4584] TS 73d0aeb063f343e9a8665162cf981f5e (127.9.221.129:44051): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5972ef6fbe684d5c92943d3bcb493fd3: Network error: Client connection negotiation failed: client connection to 127.9.221.129:44051: connect: Connection refused (error 111)
Aug 11 12:07:50 dist-test-slave-mjjr krb5kdc[7550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450067, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.130@KRBTEST.COM
I20260811 12:07:50.778064 8450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:50.762649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60374 (local address 127.9.221.130:40545)
0811 12:07:50.762863 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:50.762872 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:50.762911 (+ 39us) server_negotiation.cc:407] Connection header received
0811 12:07:50.762983 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:50.762989 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:50.763076 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:50.763199 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:50.764863 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:50.765629 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:50.766928 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:50.767322 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:50.767467 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:50.772174 (+ 4707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:50.772189 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:50.772204 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:07:50.772241 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:50.775198 (+ 2957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:50.776043 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:50.776059 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:50.776061 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:50.776186 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:50.776652 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:50.776655 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:50.776657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:50.777035 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:50.777259 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:50.777613 (+ 354us) server_negotiation.cc:299] Negotiation successful
0811 12:07:50.777780 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 12:07:50.778781 8176 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:07:50.817719 8447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:50.817144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:44051 (local address 127.0.0.1:60548)
0811 12:07:50.817408 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:50.817522 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.129:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260811 12:07:50.819010 8447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:50.818547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:44051 (local address 127.0.0.1:60560)
0811 12:07:50.818730 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:50.818855 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.129:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 12:07:50.870285 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 7969
I20260811 12:07:50.880067 8447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:50.879465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:44051 (local address 127.0.0.1:60562)
0811 12:07:50.879748 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:50.879865 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.129:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260811 12:07:50.881404 8447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:50.880858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:44051 (local address 127.0.0.1:60574)
0811 12:07:50.881123 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:50.881244 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.129:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 12:07:50.893867 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 8189
Aug 11 12:07:50 dist-test-slave-mjjr krb5kdc[7550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450067, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.131@KRBTEST.COM
I20260811 12:07:50.915135 8452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:50.894717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50070 (local address 127.9.221.131:45059)
0811 12:07:50.894930 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:50.894936 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:50.894964 (+ 28us) server_negotiation.cc:407] Connection header received
0811 12:07:50.895111 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:50.895119 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:50.895210 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:50.895338 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:50.896701 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:50.897866 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:50.899100 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:50.899811 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:50.899981 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:50.908057 (+ 8076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:50.908074 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:50.908095 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:07:50.908153 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:50.912173 (+ 4020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:50.912880 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:50.912889 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:50.912892 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:50.913069 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:50.913435 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:50.913440 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:50.913444 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:50.913916 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:50.914287 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:50.914473 (+ 186us) server_negotiation.cc:299] Negotiation successful
0811 12:07:50.914731 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260811 12:07:50.915930 8410 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:07:50.962330 8447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:50.961816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:44051 (local address 127.0.0.1:60580)
0811 12:07:50.962036 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:50.962133 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.129:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260811 12:07:50.963462 8447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:50.963069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:44051 (local address 127.0.0.1:60596)
0811 12:07:50.963261 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:50.963350 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.129:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260811 12:07:51.041335 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 8189
I20260811 12:07:51.069105 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 7575
I20260811 12:07:51.070254 7673 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:07:51.182868 8447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:51.182173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:44051 (local address 127.0.0.1:60598)
0811 12:07:51.182390 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:51.182534 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.129:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 12:07:51.183071 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 7575
2026-08-11T12:07:51Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4859 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-11T12:07:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:07:51Z Disabled control of system clock
I20260811 12:07:51.276348 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:40451
--webserver_interface=127.9.221.190
--webserver_port=0
--builtin_ntp_servers=127.9.221.148:35485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:40451
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 12:07:51.633241 8512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:07:51.633625 8512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:07:51.633737 8512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:07:51.640971 8512 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:07:51.641084 8512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:07:51.641135 8512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:07:51.641202 8512 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:07:51.641260 8512 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:07:51.650738 8512 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:35485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:40451
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:40451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.221.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:07:51.652101 8512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:07:51.654029 8512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:07:51.666308 8527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:51.666478 8530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:51.666492 8534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:07:51.667335 8512 server_base.cc:1034] running on GCE node
I20260811 12:07:51.668157 8512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:07:51.669533 8512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:07:51.670763 8512 hybrid_clock.cc:648] HybridClock initialized: now 1786450071670727 us; error 47 us; skew 500 ppm
I20260811 12:07:51.671115 8512 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:07:51.673419 8512 webserver.cc:460] Webserver started at http://127.9.221.190:39445/ using document root <none> and password file <none>
I20260811 12:07:51.674103 8512 fs_manager.cc:359] Metadata directory not provided
I20260811 12:07:51.674254 8512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:07:51.674582 8512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:07:51.677487 8512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "2e869355f76e488a8c35ecd949b97f62"
format_stamp: "Formatted at 2026-08-11 12:07:51 on dist-test-slave-mjjr"
server_key: "e6e9dd2a968fb54e7b5277059f3c7889"
server_key_iv: "b7057d82da8e9d346b7e356349cb5c14"
server_key_version: "encryptionkey@0"
I20260811 12:07:51.678323 8512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "2e869355f76e488a8c35ecd949b97f62"
format_stamp: "Formatted at 2026-08-11 12:07:51 on dist-test-slave-mjjr"
server_key: "e6e9dd2a968fb54e7b5277059f3c7889"
server_key_iv: "b7057d82da8e9d346b7e356349cb5c14"
server_key_version: "encryptionkey@0"
I20260811 12:07:51.683305 8512 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 12:07:51.686683 8543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:51.688086 8512 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260811 12:07:51.688282 8512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "2e869355f76e488a8c35ecd949b97f62"
format_stamp: "Formatted at 2026-08-11 12:07:51 on dist-test-slave-mjjr"
server_key: "e6e9dd2a968fb54e7b5277059f3c7889"
server_key_iv: "b7057d82da8e9d346b7e356349cb5c14"
server_key_version: "encryptionkey@0"
I20260811 12:07:51.688506 8512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:07:51.704928 8512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:07:51.705718 8512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:07:51.706022 8512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:07:51.725343 8512 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.190:40451
I20260811 12:07:51.725389 8636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.190:40451 every 8 connection(s)
I20260811 12:07:51.726706 8512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:07:51.730391 8640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:07:51.735666 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 8512
I20260811 12:07:51.735848 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:07:51.736405 10102 external_mini_cluster.cc:1428] Setting key ccc3f700bca59f6451785d2fb51652a3
I20260811 12:07:51.741887 8640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e869355f76e488a8c35ecd949b97f62: Bootstrap starting.
I20260811 12:07:51.744961 8640 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e869355f76e488a8c35ecd949b97f62: Neither blocks nor log segments found. Creating new log.
I20260811 12:07:51.745967 8640 log.cc:824] T 00000000000000000000000000000000 P 2e869355f76e488a8c35ecd949b97f62: Log is configured to *not* fsync() on all Append() calls
I20260811 12:07:51.749329 8640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e869355f76e488a8c35ecd949b97f62: No bootstrap required, opened a new log
I20260811 12:07:51.750691 8644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:51.738445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33022 (local address 127.9.221.190:40451)
0811 12:07:51.739581 (+ 1136us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:51.739592 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:51.739712 (+ 120us) server_negotiation.cc:407] Connection header received
0811 12:07:51.741436 (+ 1724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:51.741467 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:51.741889 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:51.742314 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:51.742854 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:51.744233 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:51.745725 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:51.746514 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:51.746724 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:51.747983 (+ 1259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:51.747995 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:51.748012 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:07:51.748720 (+ 708us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:51.748819 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:07:51.748845 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:51.748969 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:51.749166 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 12:07:51.749492 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":689,"thread_start_us":454,"threads_started":1}
I20260811 12:07:51.757836 8640 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e869355f76e488a8c35ecd949b97f62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e869355f76e488a8c35ecd949b97f62" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40451 } }
I20260811 12:07:51.758175 8640 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e869355f76e488a8c35ecd949b97f62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:07:51.758245 8640 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e869355f76e488a8c35ecd949b97f62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e869355f76e488a8c35ecd949b97f62, State: Initialized, Role: FOLLOWER
I20260811 12:07:51.758772 8640 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e869355f76e488a8c35ecd949b97f62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e869355f76e488a8c35ecd949b97f62" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40451 } }
I20260811 12:07:51.758984 8640 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e869355f76e488a8c35ecd949b97f62 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:07:51.759081 8640 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e869355f76e488a8c35ecd949b97f62 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:07:51.759209 8640 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e869355f76e488a8c35ecd949b97f62 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:07:51.761417 8640 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e869355f76e488a8c35ecd949b97f62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e869355f76e488a8c35ecd949b97f62" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40451 } }
I20260811 12:07:51.761861 8640 leader_election.cc:302] T 00000000000000000000000000000000 P 2e869355f76e488a8c35ecd949b97f62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e869355f76e488a8c35ecd949b97f62; no voters:
I20260811 12:07:51.762543 8640 leader_election.cc:288] T 00000000000000000000000000000000 P 2e869355f76e488a8c35ecd949b97f62 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:07:51.762662 8649 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e869355f76e488a8c35ecd949b97f62 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:07:51.763947 8649 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e869355f76e488a8c35ecd949b97f62 [term 1 LEADER]: Becoming Leader. State: Replica: 2e869355f76e488a8c35ecd949b97f62, State: Running, Role: LEADER
I20260811 12:07:51.764534 8649 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e869355f76e488a8c35ecd949b97f62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e869355f76e488a8c35ecd949b97f62" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40451 } }
I20260811 12:07:51.764897 8640 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e869355f76e488a8c35ecd949b97f62 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:07:51.767395 8650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e869355f76e488a8c35ecd949b97f62 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e869355f76e488a8c35ecd949b97f62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e869355f76e488a8c35ecd949b97f62" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40451 } } }
I20260811 12:07:51.767493 8654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e869355f76e488a8c35ecd949b97f62 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e869355f76e488a8c35ecd949b97f62. Latest consensus state: current_term: 1 leader_uuid: "2e869355f76e488a8c35ecd949b97f62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e869355f76e488a8c35ecd949b97f62" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40451 } } }
I20260811 12:07:51.767834 8650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e869355f76e488a8c35ecd949b97f62 [sys.catalog]: This master's current role is: LEADER
I20260811 12:07:51.767942 8654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e869355f76e488a8c35ecd949b97f62 [sys.catalog]: This master's current role is: LEADER
I20260811 12:07:51.768782 8661 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:07:51.776433 8661 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:07:51.787537 8661 catalog_manager.cc:1347] Generated new cluster ID: 6afd97fbd55f41e29c054ae4398eaa4c
I20260811 12:07:51.787678 8661 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:07:51.797490 8661 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:07:51.798713 8661 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:07:51.809726 8661 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2e869355f76e488a8c35ecd949b97f62: Generated new TSK 0
I20260811 12:07:51.810532 8661 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:07:51.823714 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.129:0
--local_ip_for_outbound_sockets=127.9.221.129
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:40451
--builtin_ntp_servers=127.9.221.148:35485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 12:07:52.172288 8685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:07:52.172570 8685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:07:52.172716 8685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:07:52.180132 8685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:07:52.180229 8685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:07:52.180434 8685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.129
I20260811 12:07:52.189668 8685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:35485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.221.129
--webserver_port=0
--tserver_master_addrs=127.9.221.190:40451
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.129
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:07:52.190941 8685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:07:52.192885 8685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:07:52.206087 8693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:52.206135 8698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:52.206175 8692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:07:52.206857 8685 server_base.cc:1034] running on GCE node
I20260811 12:07:52.207437 8685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:07:52.208390 8685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:07:52.209592 8685 hybrid_clock.cc:648] HybridClock initialized: now 1786450072209555 us; error 31 us; skew 500 ppm
I20260811 12:07:52.209899 8685 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:07:52.212266 8685 webserver.cc:460] Webserver started at http://127.9.221.129:45995/ using document root <none> and password file <none>
I20260811 12:07:52.212956 8685 fs_manager.cc:359] Metadata directory not provided
I20260811 12:07:52.213075 8685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:07:52.213416 8685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:07:52.216399 8685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-0/data/instance:
uuid: "0e5ab76134a74d9ca731247a0a0369c8"
format_stamp: "Formatted at 2026-08-11 12:07:52 on dist-test-slave-mjjr"
server_key: "2efe879dbaf8843217e106907077d8c8"
server_key_iv: "c6749c3aa07b7e5dbe67b49221b6f8dd"
server_key_version: "encryptionkey@0"
I20260811 12:07:52.217175 8685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance:
uuid: "0e5ab76134a74d9ca731247a0a0369c8"
format_stamp: "Formatted at 2026-08-11 12:07:52 on dist-test-slave-mjjr"
server_key: "2efe879dbaf8843217e106907077d8c8"
server_key_iv: "c6749c3aa07b7e5dbe67b49221b6f8dd"
server_key_version: "encryptionkey@0"
I20260811 12:07:52.222088 8685 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:07:52.225602 8709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:52.227032 8685 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260811 12:07:52.227218 8685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-0/wal
uuid: "0e5ab76134a74d9ca731247a0a0369c8"
format_stamp: "Formatted at 2026-08-11 12:07:52 on dist-test-slave-mjjr"
server_key: "2efe879dbaf8843217e106907077d8c8"
server_key_iv: "c6749c3aa07b7e5dbe67b49221b6f8dd"
server_key_version: "encryptionkey@0"
I20260811 12:07:52.227432 8685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:07:52.242769 8685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:07:52.243525 8685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:07:52.243877 8685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:07:52.244789 8685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:07:52.246222 8685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:07:52.246346 8685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:52.246495 8685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:07:52.246589 8685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:52.280879 8685 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.129:43351
I20260811 12:07:52.280925 8882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.129:43351 every 8 connection(s)
I20260811 12:07:52.282217 8685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-0/data/info.pb
I20260811 12:07:52.286818 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 8685
I20260811 12:07:52.287009 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-0/wal/instance
I20260811 12:07:52.287456 10102 external_mini_cluster.cc:1428] Setting key 04d4adb790d2ae183dcb2cba5a5df2e2
I20260811 12:07:52.290136 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.130:0
--local_ip_for_outbound_sockets=127.9.221.130
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:40451
--builtin_ntp_servers=127.9.221.148:35485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 12:07:52.295002 8885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:52.284766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.129:54515 (local address 127.9.221.190:40451)
0811 12:07:52.285240 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:52.285244 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:52.286232 (+ 988us) server_negotiation.cc:407] Connection header received
0811 12:07:52.288255 (+ 2023us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:52.288262 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:52.288360 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:52.288486 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:52.289537 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:52.290293 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:52.291994 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:52.292476 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:52.292610 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:52.294051 (+ 1441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:52.294056 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:52.294060 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:07:52.294079 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:52.294122 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:07:52.294134 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:52.294233 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:52.294562 (+ 329us) server_negotiation.cc:299] Negotiation successful
0811 12:07:52.294660 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":259,"threads_started":1}
I20260811 12:07:52.296056 8886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:52.285202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:40451 (local address 127.9.221.129:54515)
0811 12:07:52.286077 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:52.286120 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 12:07:52.287990 (+ 1870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:07:52.288718 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:07:52.288726 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:07:52.288805 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:07:52.289366 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:52.289388 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:52.290477 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:52.290481 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:52.291827 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:52.291840 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:52.292711 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:52.292720 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:52.292951 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:52.293717 (+ 766us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:07:52.293720 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:07:52.293757 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:07:52.293836 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:07:52.294247 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:07:52.294260 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:07:52.294317 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 12:07:52.294551 (+ 234us) client_negotiation.cc:240] Negotiation successful
0811 12:07:52.294966 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":267,"threads_started":1}
I20260811 12:07:52.297567 8883 heartbeater.cc:347] Connected to a master server at 127.9.221.190:40451
I20260811 12:07:52.297857 8883 heartbeater.cc:464] Registering TS with master...
I20260811 12:07:52.298560 8883 heartbeater.cc:511] Master 127.9.221.190:40451 requested a full tablet report, sending...
I20260811 12:07:52.300746 8575 ts_manager.cc:194] Registered new tserver with Master: 0e5ab76134a74d9ca731247a0a0369c8 (127.9.221.129:43351)
I20260811 12:07:52.302906 8575 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.221.129:54515
W20260811 12:07:52.629765 8891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:07:52.630096 8891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:07:52.630189 8891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:07:52.637380 8891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:07:52.637473 8891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:07:52.637575 8891 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.130
I20260811 12:07:52.646288 8891 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:35485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.221.130
--webserver_port=0
--tserver_master_addrs=127.9.221.190:40451
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.130
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:07:52.647464 8891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:07:52.649122 8891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:07:52.661854 8898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:52.662016 8896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:52.661983 8907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:07:52.662940 8891 server_base.cc:1034] running on GCE node
I20260811 12:07:52.663524 8891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:07:52.664865 8891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:07:52.666112 8891 hybrid_clock.cc:648] HybridClock initialized: now 1786450072666036 us; error 87 us; skew 500 ppm
I20260811 12:07:52.666421 8891 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:07:52.668680 8891 webserver.cc:460] Webserver started at http://127.9.221.130:35567/ using document root <none> and password file <none>
I20260811 12:07:52.669358 8891 fs_manager.cc:359] Metadata directory not provided
I20260811 12:07:52.669492 8891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:07:52.669828 8891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:07:52.672834 8891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-1/data/instance:
uuid: "d702bb812064423291ed6243ac754f90"
format_stamp: "Formatted at 2026-08-11 12:07:52 on dist-test-slave-mjjr"
server_key: "5f14fb04939791dae8d24332cdf95b24"
server_key_iv: "c7a3ae50fef8259cf10bf886f565c774"
server_key_version: "encryptionkey@0"
I20260811 12:07:52.673597 8891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance:
uuid: "d702bb812064423291ed6243ac754f90"
format_stamp: "Formatted at 2026-08-11 12:07:52 on dist-test-slave-mjjr"
server_key: "5f14fb04939791dae8d24332cdf95b24"
server_key_iv: "c7a3ae50fef8259cf10bf886f565c774"
server_key_version: "encryptionkey@0"
I20260811 12:07:52.678424 8891 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260811 12:07:52.681779 8923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:52.683012 8891 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 12:07:52.683184 8891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-1/wal
uuid: "d702bb812064423291ed6243ac754f90"
format_stamp: "Formatted at 2026-08-11 12:07:52 on dist-test-slave-mjjr"
server_key: "5f14fb04939791dae8d24332cdf95b24"
server_key_iv: "c7a3ae50fef8259cf10bf886f565c774"
server_key_version: "encryptionkey@0"
I20260811 12:07:52.683391 8891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:07:52.693435 8891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:07:52.694166 8891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:07:52.694466 8891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:07:52.695363 8891 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:07:52.696805 8891 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:07:52.696925 8891 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:52.697057 8891 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:07:52.697152 8891 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:52.730908 8891 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.130:46135
I20260811 12:07:52.730991 9092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.130:46135 every 8 connection(s)
I20260811 12:07:52.732414 8891 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-1/data/info.pb
I20260811 12:07:52.733608 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 8891
I20260811 12:07:52.733824 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-1/wal/instance
I20260811 12:07:52.734552 10102 external_mini_cluster.cc:1428] Setting key 753ed12eb9bdbbf0c2f86918e7d3710e
I20260811 12:07:52.738807 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.221.131:0
--local_ip_for_outbound_sockets=127.9.221.131
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:40451
--builtin_ntp_servers=127.9.221.148:35485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 12:07:52.756086 8885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:52.736171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:49601 (local address 127.9.221.190:40451)
0811 12:07:52.736339 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:52.736345 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:52.745361 (+ 9016us) server_negotiation.cc:407] Connection header received
0811 12:07:52.748770 (+ 3409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:52.748777 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:52.748855 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:52.748943 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:52.750605 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:52.751554 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:52.753168 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:52.753560 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:52.753693 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:52.755110 (+ 1417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:52.755114 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:52.755118 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:07:52.755154 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:52.755186 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:07:52.755198 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:52.755273 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:52.755590 (+ 317us) server_negotiation.cc:299] Negotiation successful
0811 12:07:52.755761 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 12:07:52.757122 9097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:52.735586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:40451 (local address 127.9.221.130:49601)
0811 12:07:52.745213 (+ 9627us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:52.745261 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 12:07:52.748453 (+ 3192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:07:52.749405 (+ 952us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:07:52.749416 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:07:52.749519 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:07:52.750398 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:52.750421 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:52.751754 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:52.751759 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:52.753010 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:52.753022 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:52.753781 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:52.753788 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:52.754006 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:52.754752 (+ 746us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:07:52.754757 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:07:52.754809 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:07:52.754915 (+ 106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:07:52.755301 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:07:52.755313 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:07:52.755322 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 12:07:52.755524 (+ 202us) client_negotiation.cc:240] Negotiation successful
0811 12:07:52.755976 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9025,"thread_start_us":9671,"threads_started":1}
I20260811 12:07:52.758690 9093 heartbeater.cc:347] Connected to a master server at 127.9.221.190:40451
I20260811 12:07:52.758998 9093 heartbeater.cc:464] Registering TS with master...
I20260811 12:07:52.759763 9093 heartbeater.cc:511] Master 127.9.221.190:40451 requested a full tablet report, sending...
I20260811 12:07:52.761147 8575 ts_manager.cc:194] Registered new tserver with Master: d702bb812064423291ed6243ac754f90 (127.9.221.130:46135)
I20260811 12:07:52.761996 8575 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.221.130:49601
W20260811 12:07:53.091986 9099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:07:53.092367 9099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:07:53.092535 9099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:07:53.099700 9099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:07:53.099836 9099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:07:53.100021 9099 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.221.131
I20260811 12:07:53.108796 9099 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:35485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.221.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.221.131
--webserver_port=0
--tserver_master_addrs=127.9.221.190:40451
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.221.131
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:07:53.110062 9099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:07:53.111842 9099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:07:53.124343 9105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:53.124449 9111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:53.124565 9107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:07:53.125566 9099 server_base.cc:1034] running on GCE node
I20260811 12:07:53.126199 9099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:07:53.127070 9099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:07:53.128279 9099 hybrid_clock.cc:648] HybridClock initialized: now 1786450073128224 us; error 35 us; skew 500 ppm
I20260811 12:07:53.128587 9099 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:07:53.130885 9099 webserver.cc:460] Webserver started at http://127.9.221.131:33687/ using document root <none> and password file <none>
I20260811 12:07:53.131600 9099 fs_manager.cc:359] Metadata directory not provided
I20260811 12:07:53.131805 9099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:07:53.132131 9099 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:07:53.134898 9099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-2/data/instance:
uuid: "bf8ce0edf79f435b9ffc21ec934e499e"
format_stamp: "Formatted at 2026-08-11 12:07:53 on dist-test-slave-mjjr"
server_key: "6812c0a04f5c7fdd67faecc03da1e25b"
server_key_iv: "b6432f34cadb6b12c19cb45437f46f32"
server_key_version: "encryptionkey@0"
I20260811 12:07:53.135766 9099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance:
uuid: "bf8ce0edf79f435b9ffc21ec934e499e"
format_stamp: "Formatted at 2026-08-11 12:07:53 on dist-test-slave-mjjr"
server_key: "6812c0a04f5c7fdd67faecc03da1e25b"
server_key_iv: "b6432f34cadb6b12c19cb45437f46f32"
server_key_version: "encryptionkey@0"
I20260811 12:07:53.140686 9099 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260811 12:07:53.144181 9120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:53.145530 9099 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 12:07:53.145709 9099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-2/wal
uuid: "bf8ce0edf79f435b9ffc21ec934e499e"
format_stamp: "Formatted at 2026-08-11 12:07:53 on dist-test-slave-mjjr"
server_key: "6812c0a04f5c7fdd67faecc03da1e25b"
server_key_iv: "b6432f34cadb6b12c19cb45437f46f32"
server_key_version: "encryptionkey@0"
I20260811 12:07:53.145931 9099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:07:53.162945 9099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:07:53.163830 9099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:07:53.164152 9099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:07:53.165237 9099 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:07:53.166740 9099 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:07:53.166868 9099 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:53.167007 9099 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:07:53.167104 9099 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:53.201160 9099 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.131:46151
I20260811 12:07:53.201222 9320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.131:46151 every 8 connection(s)
I20260811 12:07:53.202697 9099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-2/data/info.pb
I20260811 12:07:53.206369 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 9099
I20260811 12:07:53.206573 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-2/wal/instance
I20260811 12:07:53.207185 10102 external_mini_cluster.cc:1428] Setting key 4238ea8a657655f74dd0c6ea178bc871
I20260811 12:07:53.215938 8885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:53.205251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.131:45457 (local address 127.9.221.190:40451)
0811 12:07:53.205448 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:53.205453 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:53.207005 (+ 1552us) server_negotiation.cc:407] Connection header received
0811 12:07:53.209474 (+ 2469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:53.209483 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:53.209568 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:53.209687 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:53.210873 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:53.211611 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:53.213223 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:53.213708 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:53.213809 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:53.215168 (+ 1359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:53.215173 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:53.215176 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:07:53.215193 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:53.215220 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:07:53.215231 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:53.215298 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:53.215542 (+ 244us) server_negotiation.cc:299] Negotiation successful
0811 12:07:53.215624 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 12:07:53.216837 9324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:53.205698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:40451 (local address 127.9.221.131:45457)
0811 12:07:53.206817 (+ 1119us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:53.206867 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 12:07:53.209199 (+ 2332us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:07:53.209882 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:07:53.209911 (+ 29us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:07:53.210026 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:07:53.210693 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:53.210711 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:53.211797 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:53.211804 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:53.213088 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:53.213100 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:53.213956 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:53.213966 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:53.214197 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:53.214902 (+ 705us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:07:53.214904 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:07:53.214940 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:07:53.215014 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:07:53.215317 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:07:53.215327 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:07:53.215332 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 12:07:53.215469 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 12:07:53.215849 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":325,"threads_started":1}
I20260811 12:07:53.218005 9321 heartbeater.cc:347] Connected to a master server at 127.9.221.190:40451
I20260811 12:07:53.218282 9321 heartbeater.cc:464] Registering TS with master...
I20260811 12:07:53.219000 9321 heartbeater.cc:511] Master 127.9.221.190:40451 requested a full tablet report, sending...
I20260811 12:07:53.220420 8575 ts_manager.cc:194] Registered new tserver with Master: bf8ce0edf79f435b9ffc21ec934e499e (127.9.221.131:46151)
I20260811 12:07:53.221230 8575 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.221.131:45457
I20260811 12:07:53.222561 10102 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:07:53.232247 8885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:53.226717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33026 (local address 127.9.221.190:40451)
0811 12:07:53.226912 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:53.226916 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:53.227382 (+ 466us) server_negotiation.cc:407] Connection header received
0811 12:07:53.227561 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:53.227565 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:53.227637 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:53.227792 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:53.228210 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:53.228883 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:53.230160 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:53.230544 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:53.230650 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:53.231672 (+ 1022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:53.231681 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:53.231687 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:07:53.231708 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:53.231745 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:07:53.231758 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:53.231853 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:53.231979 (+ 126us) server_negotiation.cc:299] Negotiation successful
0811 12:07:53.232046 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 12:07:53.235883 8575 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33026:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:07:53.242517 8575 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:07:53.265425 9346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:53.254282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56062 (local address 127.9.221.129:43351)
0811 12:07:53.256068 (+ 1786us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:53.256081 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:53.256118 (+ 37us) server_negotiation.cc:407] Connection header received
0811 12:07:53.256236 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:53.256247 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:53.256554 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:53.257481 (+ 927us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:07:53.258075 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:53.259369 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:53.262444 (+ 3075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:53.263723 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:53.263884 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:53.264169 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:53.264858 (+ 689us) server_negotiation.cc:299] Negotiation successful
0811 12:07:53.265046 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1578,"thread_start_us":1453,"threads_started":1}
I20260811 12:07:53.265647 9344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:53.254256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:43351 (local address 127.0.0.1:56062)
0811 12:07:53.255165 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:53.255207 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 12:07:53.255377 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:07:53.256801 (+ 1424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:07:53.256811 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:07:53.256843 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:07:53.257216 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:53.257230 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:53.259508 (+ 2278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:53.259516 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:53.262227 (+ 2711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:53.262243 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:53.264287 (+ 2044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:53.264298 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:53.264537 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:53.264612 (+ 75us) client_negotiation.cc:769] Sending connection context
0811 12:07:53.264755 (+ 143us) client_negotiation.cc:240] Negotiation successful
0811 12:07:53.264973 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"mutex_wait_us":51,"thread_start_us":790,"threads_started":1}
I20260811 12:07:53.266449 9347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:53.254798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.130:46135 (local address 127.0.0.1:50272)
0811 12:07:53.256403 (+ 1605us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:53.256420 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:07:53.256530 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:07:53.257970 (+ 1440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:07:53.257978 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:07:53.258014 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:07:53.258524 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:53.258539 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:53.260497 (+ 1958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:53.260502 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:53.262689 (+ 2187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:53.262700 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:53.264393 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:53.264402 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:53.264585 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:53.264612 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 12:07:53.264755 (+ 143us) client_negotiation.cc:240] Negotiation successful
0811 12:07:53.264973 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1453,"mutex_wait_us":25,"thread_start_us":1304,"threads_started":1}
I20260811 12:07:53.266675 9348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:53.255208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50272 (local address 127.9.221.130:46135)
0811 12:07:53.257062 (+ 1854us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:53.257070 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:53.257101 (+ 31us) server_negotiation.cc:407] Connection header received
0811 12:07:53.257195 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:53.257204 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:53.257452 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:53.257641 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:07:53.258752 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:53.260362 (+ 1610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:53.262859 (+ 2497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:53.264184 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:53.264335 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:53.264541 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:53.266086 (+ 1545us) server_negotiation.cc:299] Negotiation successful
0811 12:07:53.266298 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1680,"thread_start_us":1784,"threads_started":1}
I20260811 12:07:53.269147 9350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:53.256451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.131:46151 (local address 127.0.0.1:35646)
0811 12:07:53.259963 (+ 3512us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:53.259981 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:07:53.260107 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:07:53.260695 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:07:53.260700 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:07:53.260723 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:07:53.261329 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:53.261343 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:53.263319 (+ 1976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:53.263324 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:53.266599 (+ 3275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:53.266618 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:53.268432 (+ 1814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:53.268441 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:53.268616 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:53.268648 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 12:07:53.268744 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 12:07:53.268859 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3384,"thread_start_us":3233,"threads_started":1}
I20260811 12:07:53.269296 9354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:53.257021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35646 (local address 127.9.221.131:46151)
0811 12:07:53.258826 (+ 1805us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:53.258834 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:53.260092 (+ 1258us) server_negotiation.cc:407] Connection header received
0811 12:07:53.260273 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:53.260280 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:53.260505 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:53.260695 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:07:53.261642 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:53.263175 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:53.266834 (+ 3659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:53.268216 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:53.268373 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:53.268649 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:53.268776 (+ 127us) server_negotiation.cc:299] Negotiation successful
0811 12:07:53.268937 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1651,"thread_start_us":431,"threads_started":1}
I20260811 12:07:53.270184 8781 tablet_service.cc:1467] Processing CreateTablet for tablet 219d6b7629474e43b6378f0cd3dd1fd4 (DEFAULT_TABLE table=test-table [id=08263efa56e14c9f8d82ede5bee06917]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:07:53.270210 8989 tablet_service.cc:1467] Processing CreateTablet for tablet 219d6b7629474e43b6378f0cd3dd1fd4 (DEFAULT_TABLE table=test-table [id=08263efa56e14c9f8d82ede5bee06917]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:07:53.271576 8989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 219d6b7629474e43b6378f0cd3dd1fd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:07:53.271577 8781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 219d6b7629474e43b6378f0cd3dd1fd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:07:53.275279 9205 tablet_service.cc:1467] Processing CreateTablet for tablet 219d6b7629474e43b6378f0cd3dd1fd4 (DEFAULT_TABLE table=test-table [id=08263efa56e14c9f8d82ede5bee06917]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:07:53.277340 9205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 219d6b7629474e43b6378f0cd3dd1fd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:07:53.289129 9360 tablet_bootstrap.cc:492] T 219d6b7629474e43b6378f0cd3dd1fd4 P d702bb812064423291ed6243ac754f90: Bootstrap starting.
I20260811 12:07:53.294360 9359 tablet_bootstrap.cc:492] T 219d6b7629474e43b6378f0cd3dd1fd4 P 0e5ab76134a74d9ca731247a0a0369c8: Bootstrap starting.
I20260811 12:07:53.296088 9360 tablet_bootstrap.cc:654] T 219d6b7629474e43b6378f0cd3dd1fd4 P d702bb812064423291ed6243ac754f90: Neither blocks nor log segments found. Creating new log.
I20260811 12:07:53.296370 9361 tablet_bootstrap.cc:492] T 219d6b7629474e43b6378f0cd3dd1fd4 P bf8ce0edf79f435b9ffc21ec934e499e: Bootstrap starting.
I20260811 12:07:53.297597 9360 log.cc:824] T 219d6b7629474e43b6378f0cd3dd1fd4 P d702bb812064423291ed6243ac754f90: Log is configured to *not* fsync() on all Append() calls
I20260811 12:07:53.298173 9359 tablet_bootstrap.cc:654] T 219d6b7629474e43b6378f0cd3dd1fd4 P 0e5ab76134a74d9ca731247a0a0369c8: Neither blocks nor log segments found. Creating new log.
I20260811 12:07:53.299346 9359 log.cc:824] T 219d6b7629474e43b6378f0cd3dd1fd4 P 0e5ab76134a74d9ca731247a0a0369c8: Log is configured to *not* fsync() on all Append() calls
I20260811 12:07:53.299698 9361 tablet_bootstrap.cc:654] T 219d6b7629474e43b6378f0cd3dd1fd4 P bf8ce0edf79f435b9ffc21ec934e499e: Neither blocks nor log segments found. Creating new log.
I20260811 12:07:53.300863 9361 log.cc:824] T 219d6b7629474e43b6378f0cd3dd1fd4 P bf8ce0edf79f435b9ffc21ec934e499e: Log is configured to *not* fsync() on all Append() calls
I20260811 12:07:53.301034 9360 tablet_bootstrap.cc:492] T 219d6b7629474e43b6378f0cd3dd1fd4 P d702bb812064423291ed6243ac754f90: No bootstrap required, opened a new log
I20260811 12:07:53.301306 9360 ts_tablet_manager.cc:1397] T 219d6b7629474e43b6378f0cd3dd1fd4 P d702bb812064423291ed6243ac754f90: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.000s
I20260811 12:07:53.302222 9359 tablet_bootstrap.cc:492] T 219d6b7629474e43b6378f0cd3dd1fd4 P 0e5ab76134a74d9ca731247a0a0369c8: No bootstrap required, opened a new log
I20260811 12:07:53.302489 9359 ts_tablet_manager.cc:1397] T 219d6b7629474e43b6378f0cd3dd1fd4 P 0e5ab76134a74d9ca731247a0a0369c8: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 12:07:53.304397 9361 tablet_bootstrap.cc:492] T 219d6b7629474e43b6378f0cd3dd1fd4 P bf8ce0edf79f435b9ffc21ec934e499e: No bootstrap required, opened a new log
I20260811 12:07:53.304646 9361 ts_tablet_manager.cc:1397] T 219d6b7629474e43b6378f0cd3dd1fd4 P bf8ce0edf79f435b9ffc21ec934e499e: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.001s
I20260811 12:07:53.305683 8883 heartbeater.cc:503] Master 127.9.221.190:40451 was elected leader, sending a full tablet report...
I20260811 12:07:53.311746 9360 raft_consensus.cc:348] T 219d6b7629474e43b6378f0cd3dd1fd4 P d702bb812064423291ed6243ac754f90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e5ab76134a74d9ca731247a0a0369c8" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43351 } } peers { permanent_uuid: "d702bb812064423291ed6243ac754f90" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46135 } } peers { permanent_uuid: "bf8ce0edf79f435b9ffc21ec934e499e" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 46151 } }
I20260811 12:07:53.311774 9359 raft_consensus.cc:348] T 219d6b7629474e43b6378f0cd3dd1fd4 P 0e5ab76134a74d9ca731247a0a0369c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e5ab76134a74d9ca731247a0a0369c8" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43351 } } peers { permanent_uuid: "d702bb812064423291ed6243ac754f90" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46135 } } peers { permanent_uuid: "bf8ce0edf79f435b9ffc21ec934e499e" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 46151 } }
I20260811 12:07:53.312217 9360 raft_consensus.cc:374] T 219d6b7629474e43b6378f0cd3dd1fd4 P d702bb812064423291ed6243ac754f90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:07:53.312220 9359 raft_consensus.cc:374] T 219d6b7629474e43b6378f0cd3dd1fd4 P 0e5ab76134a74d9ca731247a0a0369c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:07:53.312302 9360 raft_consensus.cc:729] T 219d6b7629474e43b6378f0cd3dd1fd4 P d702bb812064423291ed6243ac754f90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d702bb812064423291ed6243ac754f90, State: Initialized, Role: FOLLOWER
I20260811 12:07:53.312335 9359 raft_consensus.cc:729] T 219d6b7629474e43b6378f0cd3dd1fd4 P 0e5ab76134a74d9ca731247a0a0369c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e5ab76134a74d9ca731247a0a0369c8, State: Initialized, Role: FOLLOWER
I20260811 12:07:53.312906 9360 consensus_queue.cc:260] T 219d6b7629474e43b6378f0cd3dd1fd4 P d702bb812064423291ed6243ac754f90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e5ab76134a74d9ca731247a0a0369c8" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43351 } } peers { permanent_uuid: "d702bb812064423291ed6243ac754f90" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46135 } } peers { permanent_uuid: "bf8ce0edf79f435b9ffc21ec934e499e" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 46151 } }
I20260811 12:07:53.312997 9359 consensus_queue.cc:260] T 219d6b7629474e43b6378f0cd3dd1fd4 P 0e5ab76134a74d9ca731247a0a0369c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e5ab76134a74d9ca731247a0a0369c8" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43351 } } peers { permanent_uuid: "d702bb812064423291ed6243ac754f90" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46135 } } peers { permanent_uuid: "bf8ce0edf79f435b9ffc21ec934e499e" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 46151 } }
I20260811 12:07:53.314248 9093 heartbeater.cc:503] Master 127.9.221.190:40451 was elected leader, sending a full tablet report...
I20260811 12:07:53.314757 9360 ts_tablet_manager.cc:1428] T 219d6b7629474e43b6378f0cd3dd1fd4 P d702bb812064423291ed6243ac754f90: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260811 12:07:53.315099 9361 raft_consensus.cc:348] T 219d6b7629474e43b6378f0cd3dd1fd4 P bf8ce0edf79f435b9ffc21ec934e499e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e5ab76134a74d9ca731247a0a0369c8" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43351 } } peers { permanent_uuid: "d702bb812064423291ed6243ac754f90" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46135 } } peers { permanent_uuid: "bf8ce0edf79f435b9ffc21ec934e499e" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 46151 } }
I20260811 12:07:53.315553 9361 raft_consensus.cc:374] T 219d6b7629474e43b6378f0cd3dd1fd4 P bf8ce0edf79f435b9ffc21ec934e499e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:07:53.315681 9361 raft_consensus.cc:729] T 219d6b7629474e43b6378f0cd3dd1fd4 P bf8ce0edf79f435b9ffc21ec934e499e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf8ce0edf79f435b9ffc21ec934e499e, State: Initialized, Role: FOLLOWER
I20260811 12:07:53.316378 9361 consensus_queue.cc:260] T 219d6b7629474e43b6378f0cd3dd1fd4 P bf8ce0edf79f435b9ffc21ec934e499e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e5ab76134a74d9ca731247a0a0369c8" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43351 } } peers { permanent_uuid: "d702bb812064423291ed6243ac754f90" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46135 } } peers { permanent_uuid: "bf8ce0edf79f435b9ffc21ec934e499e" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 46151 } }
I20260811 12:07:53.316578 9359 ts_tablet_manager.cc:1428] T 219d6b7629474e43b6378f0cd3dd1fd4 P 0e5ab76134a74d9ca731247a0a0369c8: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260811 12:07:53.319139 9321 heartbeater.cc:503] Master 127.9.221.190:40451 was elected leader, sending a full tablet report...
I20260811 12:07:53.319842 9361 ts_tablet_manager.cc:1428] T 219d6b7629474e43b6378f0cd3dd1fd4 P bf8ce0edf79f435b9ffc21ec934e499e: Time spent starting tablet: real 0.015s user 0.008s sys 0.008s
I20260811 12:07:53.403482 9370 raft_consensus.cc:482] T 219d6b7629474e43b6378f0cd3dd1fd4 P d702bb812064423291ed6243ac754f90 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:07:53.403817 9370 raft_consensus.cc:504] T 219d6b7629474e43b6378f0cd3dd1fd4 P d702bb812064423291ed6243ac754f90 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e5ab76134a74d9ca731247a0a0369c8" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43351 } } peers { permanent_uuid: "d702bb812064423291ed6243ac754f90" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46135 } } peers { permanent_uuid: "bf8ce0edf79f435b9ffc21ec934e499e" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 46151 } }
I20260811 12:07:53.405409 9370 leader_election.cc:288] T 219d6b7629474e43b6378f0cd3dd1fd4 P d702bb812064423291ed6243ac754f90 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e5ab76134a74d9ca731247a0a0369c8 (127.9.221.129:43351), bf8ce0edf79f435b9ffc21ec934e499e (127.9.221.131:46151)
I20260811 12:07:53.413431 9346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:53.405950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:49205 (local address 127.9.221.129:43351)
0811 12:07:53.406128 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:53.406136 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:53.406857 (+ 721us) server_negotiation.cc:407] Connection header received
0811 12:07:53.407072 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:53.407077 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:53.407171 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:53.407268 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:07:53.408058 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:53.409202 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:53.411558 (+ 2356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:53.412430 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:53.412594 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:53.412676 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:53.413022 (+ 346us) server_negotiation.cc:299] Negotiation successful
0811 12:07:53.413163 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 12:07:53.413429 9354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:53.405853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.221.130:43701 (local address 127.9.221.131:46151)
0811 12:07:53.406075 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:53.406081 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:53.406923 (+ 842us) server_negotiation.cc:407] Connection header received
0811 12:07:53.407072 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:53.407077 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:53.407164 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:53.407269 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:07:53.408092 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:53.409223 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:53.411606 (+ 2383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:53.412453 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:53.412647 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:53.412722 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:53.413084 (+ 362us) server_negotiation.cc:299] Negotiation successful
0811 12:07:53.413213 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 12:07:53.413482 9374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:53.405747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:43351 (local address 127.9.221.130:49205)
0811 12:07:53.406727 (+ 980us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:53.406747 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:07:53.406925 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:07:53.407301 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:07:53.407306 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:07:53.407343 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:07:53.407886 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:53.407901 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:53.409354 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:53.409362 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:53.411343 (+ 1981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:53.411357 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:53.412633 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:53.412639 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:53.412852 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:53.412887 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 12:07:53.412961 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 12:07:53.413105 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":797,"mutex_wait_us":125,"thread_start_us":546,"threads_started":1}
I20260811 12:07:53.413573 9375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:53.405996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.131:46151 (local address 127.9.221.130:43701)
0811 12:07:53.406810 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:53.406831 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:07:53.406935 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:07:53.407301 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:07:53.407306 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:07:53.407398 (+ 92us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:07:53.407932 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:53.407944 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:53.409426 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:53.409432 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:53.411352 (+ 1920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:07:53.411365 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:07:53.412686 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:07:53.412695 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:07:53.412901 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:53.412924 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 12:07:53.413041 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 12:07:53.413162 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":695,"mutex_wait_us":184,"spinlock_wait_cycles":21888,"thread_start_us":471,"threads_started":1}
I20260811 12:07:53.414773 9242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "219d6b7629474e43b6378f0cd3dd1fd4" candidate_uuid: "d702bb812064423291ed6243ac754f90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf8ce0edf79f435b9ffc21ec934e499e" is_pre_election: true
I20260811 12:07:53.414829 8812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "219d6b7629474e43b6378f0cd3dd1fd4" candidate_uuid: "d702bb812064423291ed6243ac754f90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e5ab76134a74d9ca731247a0a0369c8" is_pre_election: true
I20260811 12:07:53.415302 9242 raft_consensus.cc:2454] T 219d6b7629474e43b6378f0cd3dd1fd4 P bf8ce0edf79f435b9ffc21ec934e499e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d702bb812064423291ed6243ac754f90 in term 0.
I20260811 12:07:53.415343 8812 raft_consensus.cc:2454] T 219d6b7629474e43b6378f0cd3dd1fd4 P 0e5ab76134a74d9ca731247a0a0369c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d702bb812064423291ed6243ac754f90 in term 0.
I20260811 12:07:53.415962 8928 leader_election.cc:302] T 219d6b7629474e43b6378f0cd3dd1fd4 P d702bb812064423291ed6243ac754f90 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bf8ce0edf79f435b9ffc21ec934e499e, d702bb812064423291ed6243ac754f90; no voters:
I20260811 12:07:53.416338 9370 raft_consensus.cc:2781] T 219d6b7629474e43b6378f0cd3dd1fd4 P d702bb812064423291ed6243ac754f90 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:07:53.416495 9370 raft_consensus.cc:482] T 219d6b7629474e43b6378f0cd3dd1fd4 P d702bb812064423291ed6243ac754f90 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:07:53.416628 9370 raft_consensus.cc:3037] T 219d6b7629474e43b6378f0cd3dd1fd4 P d702bb812064423291ed6243ac754f90 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:07:53.418826 9370 raft_consensus.cc:504] T 219d6b7629474e43b6378f0cd3dd1fd4 P d702bb812064423291ed6243ac754f90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e5ab76134a74d9ca731247a0a0369c8" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43351 } } peers { permanent_uuid: "d702bb812064423291ed6243ac754f90" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46135 } } peers { permanent_uuid: "bf8ce0edf79f435b9ffc21ec934e499e" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 46151 } }
I20260811 12:07:53.419449 9370 leader_election.cc:288] T 219d6b7629474e43b6378f0cd3dd1fd4 P d702bb812064423291ed6243ac754f90 [CANDIDATE]: Term 1 election: Requested vote from peers 0e5ab76134a74d9ca731247a0a0369c8 (127.9.221.129:43351), bf8ce0edf79f435b9ffc21ec934e499e (127.9.221.131:46151)
I20260811 12:07:53.419970 8812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "219d6b7629474e43b6378f0cd3dd1fd4" candidate_uuid: "d702bb812064423291ed6243ac754f90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e5ab76134a74d9ca731247a0a0369c8"
I20260811 12:07:53.419982 9242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "219d6b7629474e43b6378f0cd3dd1fd4" candidate_uuid: "d702bb812064423291ed6243ac754f90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf8ce0edf79f435b9ffc21ec934e499e"
I20260811 12:07:53.420176 9242 raft_consensus.cc:3037] T 219d6b7629474e43b6378f0cd3dd1fd4 P bf8ce0edf79f435b9ffc21ec934e499e [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:07:53.420167 8812 raft_consensus.cc:3037] T 219d6b7629474e43b6378f0cd3dd1fd4 P 0e5ab76134a74d9ca731247a0a0369c8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:07:53.422365 8812 raft_consensus.cc:2454] T 219d6b7629474e43b6378f0cd3dd1fd4 P 0e5ab76134a74d9ca731247a0a0369c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d702bb812064423291ed6243ac754f90 in term 1.
I20260811 12:07:53.422531 9242 raft_consensus.cc:2454] T 219d6b7629474e43b6378f0cd3dd1fd4 P bf8ce0edf79f435b9ffc21ec934e499e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d702bb812064423291ed6243ac754f90 in term 1.
I20260811 12:07:53.422905 8926 leader_election.cc:302] T 219d6b7629474e43b6378f0cd3dd1fd4 P d702bb812064423291ed6243ac754f90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e5ab76134a74d9ca731247a0a0369c8, d702bb812064423291ed6243ac754f90; no voters:
I20260811 12:07:53.423261 9370 raft_consensus.cc:2781] T 219d6b7629474e43b6378f0cd3dd1fd4 P d702bb812064423291ed6243ac754f90 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:07:53.423744 9370 raft_consensus.cc:686] T 219d6b7629474e43b6378f0cd3dd1fd4 P d702bb812064423291ed6243ac754f90 [term 1 LEADER]: Becoming Leader. State: Replica: d702bb812064423291ed6243ac754f90, State: Running, Role: LEADER
I20260811 12:07:53.424512 9370 consensus_queue.cc:237] T 219d6b7629474e43b6378f0cd3dd1fd4 P d702bb812064423291ed6243ac754f90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e5ab76134a74d9ca731247a0a0369c8" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43351 } } peers { permanent_uuid: "d702bb812064423291ed6243ac754f90" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46135 } } peers { permanent_uuid: "bf8ce0edf79f435b9ffc21ec934e499e" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 46151 } }
I20260811 12:07:53.428932 8575 catalog_manager.cc:5495] T 219d6b7629474e43b6378f0cd3dd1fd4 P d702bb812064423291ed6243ac754f90 reported cstate change: term changed from 0 to 1, leader changed from <none> to d702bb812064423291ed6243ac754f90 (127.9.221.130). New cstate: current_term: 1 leader_uuid: "d702bb812064423291ed6243ac754f90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e5ab76134a74d9ca731247a0a0369c8" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d702bb812064423291ed6243ac754f90" member_type: VOTER last_known_addr { host: "127.9.221.130" port: 46135 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf8ce0edf79f435b9ffc21ec934e499e" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 46151 } health_report { overall_health: UNKNOWN } } }
W20260811 12:07:53.454449 9323 tablet.cc:2369] T 219d6b7629474e43b6378f0cd3dd1fd4 P bf8ce0edf79f435b9ffc21ec934e499e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:07:53.458778 9348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:53.453382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50276 (local address 127.9.221.130:46135)
0811 12:07:53.453598 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:53.453607 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:53.453703 (+ 96us) server_negotiation.cc:407] Connection header received
0811 12:07:53.453868 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:53.453872 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:53.453958 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:53.454079 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:07:53.454535 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:53.455301 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:53.456669 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:53.457094 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:53.457189 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:53.457737 (+ 548us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:07:53.458209 (+ 472us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:07:53.458315 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:53.458411 (+ 96us) server_negotiation.cc:299] Negotiation successful
0811 12:07:53.458471 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 12:07:53.467083 8812 raft_consensus.cc:1261] T 219d6b7629474e43b6378f0cd3dd1fd4 P 0e5ab76134a74d9ca731247a0a0369c8 [term 1 FOLLOWER]: Refusing update from remote peer d702bb812064423291ed6243ac754f90: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:07:53.467083 9242 raft_consensus.cc:1261] T 219d6b7629474e43b6378f0cd3dd1fd4 P bf8ce0edf79f435b9ffc21ec934e499e [term 1 FOLLOWER]: Refusing update from remote peer d702bb812064423291ed6243ac754f90: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:07:53.468250 9370 consensus_queue.cc:1035] T 219d6b7629474e43b6378f0cd3dd1fd4 P d702bb812064423291ed6243ac754f90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf8ce0edf79f435b9ffc21ec934e499e" member_type: VOTER last_known_addr { host: "127.9.221.131" port: 46151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:07:53.468597 9377 consensus_queue.cc:1035] T 219d6b7629474e43b6378f0cd3dd1fd4 P d702bb812064423291ed6243ac754f90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e5ab76134a74d9ca731247a0a0369c8" member_type: VOTER last_known_addr { host: "127.9.221.129" port: 43351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:07:53.479355 9382 mvcc.cc:205] Tried to move back new op lower bound from 7317299500909875200 to 7317299500750700544. Current Snapshot: MvccSnapshot[applied={T|T < 7317299500909875200}]
I20260811 12:07:53.497130 8575 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33026:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:07:53.497359 8575 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33026:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:07:53.500749 8575 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2e869355f76e488a8c35ecd949b97f62: Sending DeleteTablet for 3 replicas of tablet 219d6b7629474e43b6378f0cd3dd1fd4
I20260811 12:07:53.501864 8781 tablet_service.cc:1514] Processing DeleteTablet for tablet 219d6b7629474e43b6378f0cd3dd1fd4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:07:53 UTC) from {username='slave'} at 127.0.0.1:56062
I20260811 12:07:53.502158 8989 tablet_service.cc:1514] Processing DeleteTablet for tablet 219d6b7629474e43b6378f0cd3dd1fd4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:07:53 UTC) from {username='slave'} at 127.0.0.1:50272
I20260811 12:07:53.502590 9205 tablet_service.cc:1514] Processing DeleteTablet for tablet 219d6b7629474e43b6378f0cd3dd1fd4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:07:53 UTC) from {username='slave'} at 127.0.0.1:35646
I20260811 12:07:53.503324 9404 tablet_replica.cc:330] T 219d6b7629474e43b6378f0cd3dd1fd4 P d702bb812064423291ed6243ac754f90: stopping tablet replica
I20260811 12:07:53.503759 9404 raft_consensus.cc:2229] T 219d6b7629474e43b6378f0cd3dd1fd4 P d702bb812064423291ed6243ac754f90 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:07:53.504436 9404 raft_consensus.cc:2258] T 219d6b7629474e43b6378f0cd3dd1fd4 P d702bb812064423291ed6243ac754f90 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:07:53.504522 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 8685
I20260811 12:07:53.505421 9402 tablet_replica.cc:330] T 219d6b7629474e43b6378f0cd3dd1fd4 P 0e5ab76134a74d9ca731247a0a0369c8: stopping tablet replica
I20260811 12:07:53.505828 9402 raft_consensus.cc:2229] T 219d6b7629474e43b6378f0cd3dd1fd4 P 0e5ab76134a74d9ca731247a0a0369c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:07:53.506359 9402 raft_consensus.cc:2258] T 219d6b7629474e43b6378f0cd3dd1fd4 P 0e5ab76134a74d9ca731247a0a0369c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:07:53.509090 9402 ts_tablet_manager.cc:1905] T 219d6b7629474e43b6378f0cd3dd1fd4 P 0e5ab76134a74d9ca731247a0a0369c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:07:53.512405 9404 ts_tablet_manager.cc:1905] T 219d6b7629474e43b6378f0cd3dd1fd4 P d702bb812064423291ed6243ac754f90: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:07:53.514652 9346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:53.505903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56068 (local address 127.9.221.129:43351)
0811 12:07:53.506094 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:53.506102 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:53.506122 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:07:53.506362 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:53.506371 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:53.506515 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:53.506601 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:53.507262 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:53.508546 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:53.510483 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:53.511198 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:53.511321 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:53.512840 (+ 1519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:53.512853 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:53.512873 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:07:53.513623 (+ 750us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:53.513681 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:07:53.513715 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:53.513824 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:53.514110 (+ 286us) server_negotiation.cc:299] Negotiation successful
0811 12:07:53.514228 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 12:07:53.515563 9405 tablet_replica.cc:330] T 219d6b7629474e43b6378f0cd3dd1fd4 P bf8ce0edf79f435b9ffc21ec934e499e: stopping tablet replica
I20260811 12:07:53.515872 8878 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:07:53.516191 9405 raft_consensus.cc:2229] T 219d6b7629474e43b6378f0cd3dd1fd4 P bf8ce0edf79f435b9ffc21ec934e499e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:07:53.516880 9405 raft_consensus.cc:2258] T 219d6b7629474e43b6378f0cd3dd1fd4 P bf8ce0edf79f435b9ffc21ec934e499e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:07:53.519275 9405 ts_tablet_manager.cc:1905] T 219d6b7629474e43b6378f0cd3dd1fd4 P bf8ce0edf79f435b9ffc21ec934e499e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:07:53.524780 9404 ts_tablet_manager.cc:1918] T 219d6b7629474e43b6378f0cd3dd1fd4 P d702bb812064423291ed6243ac754f90: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:07:53.525017 9404 log.cc:1196] T 219d6b7629474e43b6378f0cd3dd1fd4 P d702bb812064423291ed6243ac754f90: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-1/wal/wals/219d6b7629474e43b6378f0cd3dd1fd4
I20260811 12:07:53.525666 9404 ts_tablet_manager.cc:1939] T 219d6b7629474e43b6378f0cd3dd1fd4 P d702bb812064423291ed6243ac754f90: Deleting consensus metadata
I20260811 12:07:53.527586 8548 catalog_manager.cc:4841] TS d702bb812064423291ed6243ac754f90 (127.9.221.130:46135): tablet 219d6b7629474e43b6378f0cd3dd1fd4 (table test-table [id=08263efa56e14c9f8d82ede5bee06917]) successfully deleted
I20260811 12:07:53.529321 9405 ts_tablet_manager.cc:1918] T 219d6b7629474e43b6378f0cd3dd1fd4 P bf8ce0edf79f435b9ffc21ec934e499e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:07:53.529520 9405 log.cc:1196] T 219d6b7629474e43b6378f0cd3dd1fd4 P bf8ce0edf79f435b9ffc21ec934e499e: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-2/wal/wals/219d6b7629474e43b6378f0cd3dd1fd4
I20260811 12:07:53.530292 9405 ts_tablet_manager.cc:1939] T 219d6b7629474e43b6378f0cd3dd1fd4 P bf8ce0edf79f435b9ffc21ec934e499e: Deleting consensus metadata
I20260811 12:07:53.532612 8548 catalog_manager.cc:4841] TS bf8ce0edf79f435b9ffc21ec934e499e (127.9.221.131:46151): tablet 219d6b7629474e43b6378f0cd3dd1fd4 (table test-table [id=08263efa56e14c9f8d82ede5bee06917]) successfully deleted
I20260811 12:07:53.651983 9402 ts_tablet_manager.cc:1918] T 219d6b7629474e43b6378f0cd3dd1fd4 P 0e5ab76134a74d9ca731247a0a0369c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:07:53.652276 9402 log.cc:1196] T 219d6b7629474e43b6378f0cd3dd1fd4 P 0e5ab76134a74d9ca731247a0a0369c8: Deleting WAL directory at /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449743084027-10102-0/minicluster-data/ts-0/wal/wals/219d6b7629474e43b6378f0cd3dd1fd4
I20260811 12:07:53.653061 9402 ts_tablet_manager.cc:1939] T 219d6b7629474e43b6378f0cd3dd1fd4 P 0e5ab76134a74d9ca731247a0a0369c8: Deleting consensus metadata
I20260811 12:07:53.653041 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 8685
W20260811 12:07:53.680898 8547 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.221.129:43351 (error 108)
I20260811 12:07:53.681103 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 8891
I20260811 12:07:53.683132 9344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:53.682598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:43351 (local address 127.0.0.1:56078)
0811 12:07:53.682842 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:53.682952 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.129:43351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
W20260811 12:07:53.683410 8547 catalog_manager.cc:4584] TS 0e5ab76134a74d9ca731247a0a0369c8 (127.9.221.129:43351): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 219d6b7629474e43b6378f0cd3dd1fd4: Network error: Client connection negotiation failed: client connection to 127.9.221.129:43351: connect: Connection refused (error 111)
I20260811 12:07:53.689391 9348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:53.682352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50292 (local address 127.9.221.130:46135)
0811 12:07:53.682547 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:53.682552 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:53.682570 (+ 18us) server_negotiation.cc:407] Connection header received
0811 12:07:53.682630 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:53.682634 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:53.682738 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:53.682881 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:53.683447 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:53.684361 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:53.685759 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:53.686357 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:53.686546 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:53.687936 (+ 1390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:53.687945 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:53.687959 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:07:53.688570 (+ 611us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:53.688628 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:07:53.688646 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:53.688752 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:53.688934 (+ 182us) server_negotiation.cc:299] Negotiation successful
0811 12:07:53.689030 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 12:07:53.690107 9087 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:07:53.737159 9344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:53.736620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:43351 (local address 127.0.0.1:56088)
0811 12:07:53.736851 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:53.736969 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.129:43351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 12:07:53.738319 9344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:53.737901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:43351 (local address 127.0.0.1:56100)
0811 12:07:53.738072 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:53.738165 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.129:43351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 12:07:53.799268 9344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:53.798667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:43351 (local address 127.0.0.1:56116)
0811 12:07:53.798947 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:53.799058 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.129:43351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
I20260811 12:07:53.800468 9344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:53.800098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:43351 (local address 127.0.0.1:56118)
0811 12:07:53.800260 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:53.800345 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.129:43351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 12:07:53.807140 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 8891
I20260811 12:07:53.830500 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 9099
I20260811 12:07:53.837280 9354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:53.831334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35648 (local address 127.9.221.131:46151)
0811 12:07:53.831566 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:53.831571 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:53.831593 (+ 22us) server_negotiation.cc:407] Connection header received
0811 12:07:53.831850 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:53.831855 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:53.831951 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:53.832039 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:53.832435 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:53.833223 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:53.834452 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:53.834871 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:53.834974 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:53.835975 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:53.835983 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:53.835996 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:07:53.836563 (+ 567us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:53.836609 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:07:53.836629 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:53.836722 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:53.836824 (+ 102us) server_negotiation.cc:299] Negotiation successful
0811 12:07:53.836910 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 12:07:53.837944 9315 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:07:53.880970 9344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:53.880394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:43351 (local address 127.0.0.1:56128)
0811 12:07:53.880618 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:53.880766 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.129:43351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 12:07:53.882026 9344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:53.881609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.129:43351 (local address 127.0.0.1:56130)
0811 12:07:53.881814 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0811 12:07:53.881903 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.129:43351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260811 12:07:53.930510 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 9099
I20260811 12:07:53.956923 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 8512
I20260811 12:07:53.958073 8630 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:07:54.067723 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 8512
2026-08-11T12:07:54Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2886 ms)
[----------] 6 tests from EncryptionPolicyTest (23139 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449743084027-10102-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:07:54 dist-test-slave-mjjr krb5kdc[9416](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:07:54 dist-test-slave-mjjr krb5kdc[9416](info): set up 2 sockets
Aug 11 12:07:54 dist-test-slave-mjjr krb5kdc[9416](info): commencing operation
krb5kdc: starting...
W20260811 12:07:55.405985 10102 mini_kdc.cc:121] Time spent starting KDC: real 1.288s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Aug 11 12:07:55 dist-test-slave-mjjr krb5kdc[9416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450075, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:07:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:07:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449743084027-10102-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449743084027-10102-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 12:07:55.723151 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44881
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449743084027-10102-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44881
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:07:56.065342 9442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:07:56.065698 9442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:07:56.065887 9442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:07:56.073725 9442 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:07:56.073875 9442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:07:56.073962 9442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:07:56.074050 9442 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:07:56.074126 9442 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:07:56.083741 9442 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44881
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449743084027-10102-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:07:56.085093 9442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:07:56.086400 9442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:07:56.098505 9454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:56.098544 9459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:56.099251 9450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:07:56.099326 9442 server_base.cc:1034] running on GCE node
I20260811 12:07:56.100237 9442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:07:56.101575 9442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:07:56.102839 9442 hybrid_clock.cc:648] HybridClock initialized: now 1786450076102789 us; error 31 us; skew 500 ppm
I20260811 12:07:56.103196 9442 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:07:56 dist-test-slave-mjjr krb5kdc[9416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450076, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:07:56.109508 9442 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 12:07:56.111004 9442 webserver.cc:460] Webserver started at http://127.0.0.1:46161/ using document root <none> and password file <none>
I20260811 12:07:56.111706 9442 fs_manager.cc:359] Metadata directory not provided
I20260811 12:07:56.111876 9442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:07:56.112180 9442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:07:56.115044 9442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "3a9f2f3deab14220867c2c120e309172"
format_stamp: "Formatted at 2026-08-11 12:07:56 on dist-test-slave-mjjr"
server_key: "a25f2ac67a461a54186c0f0c41d9fc96"
server_key_iv: "a4d43dfa17776f943130e37a73220a9e"
server_key_version: "encryptionkey@0"
I20260811 12:07:56.115900 9442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "3a9f2f3deab14220867c2c120e309172"
format_stamp: "Formatted at 2026-08-11 12:07:56 on dist-test-slave-mjjr"
server_key: "a25f2ac67a461a54186c0f0c41d9fc96"
server_key_iv: "a4d43dfa17776f943130e37a73220a9e"
server_key_version: "encryptionkey@0"
I20260811 12:07:56.120743 9442 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260811 12:07:56.124126 9467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:56.125554 9442 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 12:07:56.125764 9442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "3a9f2f3deab14220867c2c120e309172"
format_stamp: "Formatted at 2026-08-11 12:07:56 on dist-test-slave-mjjr"
server_key: "a25f2ac67a461a54186c0f0c41d9fc96"
server_key_iv: "a4d43dfa17776f943130e37a73220a9e"
server_key_version: "encryptionkey@0"
I20260811 12:07:56.126008 9442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:07:56.162039 9442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:07:56.166716 9442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:07:56.167032 9442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:07:56.186156 9442 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44881
I20260811 12:07:56.186195 9540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44881 every 8 connection(s)
I20260811 12:07:56.187558 9442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:07:56.191174 9542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:07:56.192183 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 9442
I20260811 12:07:56.192420 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:07:56.193293 10102 external_mini_cluster.cc:1428] Setting key 887500ec506c307e324625266bf3d6bc
I20260811 12:07:56.205360 9542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a9f2f3deab14220867c2c120e309172: Bootstrap starting.
Aug 11 12:07:56 dist-test-slave-mjjr krb5kdc[9416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450075, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 12:07:56.209023 9542 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a9f2f3deab14220867c2c120e309172: Neither blocks nor log segments found. Creating new log.
I20260811 12:07:56.210115 9542 log.cc:824] T 00000000000000000000000000000000 P 3a9f2f3deab14220867c2c120e309172: Log is configured to *not* fsync() on all Append() calls
I20260811 12:07:56.213217 9542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a9f2f3deab14220867c2c120e309172: No bootstrap required, opened a new log
I20260811 12:07:56.216935 9546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:56.195266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42329 (local address 127.0.0.1:44881)
0811 12:07:56.196111 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:56.196122 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:56.196165 (+ 43us) server_negotiation.cc:407] Connection header received
0811 12:07:56.197717 (+ 1552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:56.197737 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:56.198101 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:56.198420 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:56.200284 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:56.201579 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:56.203515 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:56.204231 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:56.204362 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:56.209243 (+ 4881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:56.209252 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:56.209264 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:07:56.209299 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:56.212728 (+ 3429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:56.213598 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:56.213604 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:56.213610 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:56.213787 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:56.214272 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:56.214275 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:56.214277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:56.214766 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:56.215131 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:56.215674 (+ 543us) server_negotiation.cc:299] Negotiation successful
0811 12:07:56.215986 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":625,"thread_start_us":400,"threads_started":1}
I20260811 12:07:56.220791 10102 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:07:56.221400 9542 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a9f2f3deab14220867c2c120e309172 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a9f2f3deab14220867c2c120e309172" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44881 } }
I20260811 12:07:56.221741 9542 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3a9f2f3deab14220867c2c120e309172 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:07:56.221846 9542 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a9f2f3deab14220867c2c120e309172 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a9f2f3deab14220867c2c120e309172, State: Initialized, Role: FOLLOWER
I20260811 12:07:56.222364 9542 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a9f2f3deab14220867c2c120e309172 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a9f2f3deab14220867c2c120e309172" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44881 } }
I20260811 12:07:56.222622 9542 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3a9f2f3deab14220867c2c120e309172 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:07:56.222765 9542 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3a9f2f3deab14220867c2c120e309172 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:07:56.222944 9542 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a9f2f3deab14220867c2c120e309172 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:07:56.225277 9542 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3a9f2f3deab14220867c2c120e309172 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a9f2f3deab14220867c2c120e309172" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44881 } }
I20260811 12:07:56.225832 9542 leader_election.cc:302] T 00000000000000000000000000000000 P 3a9f2f3deab14220867c2c120e309172 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3a9f2f3deab14220867c2c120e309172; no voters:
I20260811 12:07:56.226557 9542 leader_election.cc:288] T 00000000000000000000000000000000 P 3a9f2f3deab14220867c2c120e309172 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:07:56.226737 9560 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3a9f2f3deab14220867c2c120e309172 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:07:56.228581 9542 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a9f2f3deab14220867c2c120e309172 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:07:56.230378 9560 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3a9f2f3deab14220867c2c120e309172 [term 1 LEADER]: Becoming Leader. State: Replica: 3a9f2f3deab14220867c2c120e309172, State: Running, Role: LEADER
I20260811 12:07:56.231083 9560 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a9f2f3deab14220867c2c120e309172 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a9f2f3deab14220867c2c120e309172" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44881 } }
I20260811 12:07:56.234413 9561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a9f2f3deab14220867c2c120e309172 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3a9f2f3deab14220867c2c120e309172" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a9f2f3deab14220867c2c120e309172" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44881 } } }
I20260811 12:07:56.234733 9561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a9f2f3deab14220867c2c120e309172 [sys.catalog]: This master's current role is: LEADER
I20260811 12:07:56.238512 9574 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:07:56.240134 9562 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a9f2f3deab14220867c2c120e309172 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a9f2f3deab14220867c2c120e309172. Latest consensus state: current_term: 1 leader_uuid: "3a9f2f3deab14220867c2c120e309172" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a9f2f3deab14220867c2c120e309172" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44881 } } }
I20260811 12:07:56.240497 9562 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a9f2f3deab14220867c2c120e309172 [sys.catalog]: This master's current role is: LEADER
I20260811 12:07:56.247669 9546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:56.225830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45965 (local address 127.0.0.1:44881)
0811 12:07:56.226029 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:56.226035 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:56.226646 (+ 611us) server_negotiation.cc:407] Connection header received
0811 12:07:56.226848 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:56.226856 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:56.226944 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:56.227061 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:56.229673 (+ 2612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:56.231055 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:56.233095 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:56.233886 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:56.234029 (+ 143us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:56.237919 (+ 3890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:56.237932 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:56.237939 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:07:56.237991 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:56.242404 (+ 4413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:56.243711 (+ 1307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:56.243718 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:56.243724 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:56.243891 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:07:56.244662 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:07:56.244670 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:07:56.244676 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:07:56.245159 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:56.246863 (+ 1704us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:56.247081 (+ 218us) server_negotiation.cc:299] Negotiation successful
0811 12:07:56.247284 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 12:07:56.252938 9574 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:07:56.267176 9574 catalog_manager.cc:1347] Generated new cluster ID: bc2ea1ca737f458e99c5b68c18512bb0
I20260811 12:07:56.267289 9574 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:07:56.277848 9574 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:07:56.279584 9574 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:07:56.291385 9574 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3a9f2f3deab14220867c2c120e309172: Generated new TSK 0
I20260811 12:07:56.292693 9574 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:07:56.306136 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 9442
I20260811 12:07:56.307863 9532 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:07:56.398455 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 9442
W20260811 12:07:56.427160 9413 connection.cc:535] client connection to 127.0.0.1:44881 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44881): Connection reset by peer (error 104)
2026-08-11T12:07:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2329 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449743084027-10102-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:07:56 dist-test-slave-mjjr krb5kdc[9599](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:07:56 dist-test-slave-mjjr krb5kdc[9599](info): set up 2 sockets
Aug 11 12:07:56 dist-test-slave-mjjr krb5kdc[9599](info): commencing operation
krb5kdc: starting...
W20260811 12:07:57.744792 10102 mini_kdc.cc:121] Time spent starting KDC: real 1.298s user 0.005s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Aug 11 12:07:57 dist-test-slave-mjjr krb5kdc[9599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:07:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:07:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449743084027-10102-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449743084027-10102-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 12:07:58.048484 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37589
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449743084027-10102-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37589
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:07:58.395967 9616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:07:58.396317 9616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:07:58.396517 9616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:07:58.403885 9616 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:07:58.404028 9616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:07:58.404150 9616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:07:58.404269 9616 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:07:58.404364 9616 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:07:58.414243 9616 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37589
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449743084027-10102-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:07:58.415566 9616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:07:58.416988 9616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:07:58.429744 9626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:58.429689 9627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:07:58.429765 9629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:07:58.430984 9616 server_base.cc:1034] running on GCE node
I20260811 12:07:58.431773 9616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:07:58.433102 9616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:07:58.434340 9616 hybrid_clock.cc:648] HybridClock initialized: now 1786450078434300 us; error 50 us; skew 500 ppm
I20260811 12:07:58.434675 9616 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:07:58 dist-test-slave-mjjr krb5kdc[9599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450078, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:07:58.437829 9616 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 12:07:58.439388 9616 webserver.cc:460] Webserver started at http://127.0.0.1:42903/ using document root <none> and password file <none>
I20260811 12:07:58.440115 9616 fs_manager.cc:359] Metadata directory not provided
I20260811 12:07:58.440232 9616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:07:58.440497 9616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:07:58.443488 9616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "7a7885d936e94855a6f15c00fdca5cc1"
format_stamp: "Formatted at 2026-08-11 12:07:58 on dist-test-slave-mjjr"
server_key: "71590a7ad59c1f88050568ce37e03f42"
server_key_iv: "f2d3e056ce71e954eb586198b97d4e57"
server_key_version: "encryptionkey@0"
I20260811 12:07:58.444360 9616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "7a7885d936e94855a6f15c00fdca5cc1"
format_stamp: "Formatted at 2026-08-11 12:07:58 on dist-test-slave-mjjr"
server_key: "71590a7ad59c1f88050568ce37e03f42"
server_key_iv: "f2d3e056ce71e954eb586198b97d4e57"
server_key_version: "encryptionkey@0"
I20260811 12:07:58.449550 9616 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 12:07:58.453079 9641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:07:58.454361 9616 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260811 12:07:58.454515 9616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "7a7885d936e94855a6f15c00fdca5cc1"
format_stamp: "Formatted at 2026-08-11 12:07:58 on dist-test-slave-mjjr"
server_key: "71590a7ad59c1f88050568ce37e03f42"
server_key_iv: "f2d3e056ce71e954eb586198b97d4e57"
server_key_version: "encryptionkey@0"
I20260811 12:07:58.454701 9616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:07:58.476460 9616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:07:58.480486 9616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:07:58.480808 9616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:07:58.501214 9616 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37589
I20260811 12:07:58.501251 9724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37589 every 8 connection(s)
I20260811 12:07:58.502764 9616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:07:58.506448 9727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:07:58.507748 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 9616
I20260811 12:07:58.507937 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:07:58.508399 10102 external_mini_cluster.cc:1428] Setting key 5b732050ffb635a22f2f42e41dca1568
I20260811 12:07:58.519932 9727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a7885d936e94855a6f15c00fdca5cc1: Bootstrap starting.
I20260811 12:07:58.521371 9734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:58.510116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34789 (local address 127.0.0.1:37589)
0811 12:07:58.510948 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:58.510958 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:58.511001 (+ 43us) server_negotiation.cc:407] Connection header received
0811 12:07:58.512606 (+ 1605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:58.512627 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:58.512997 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:58.513315 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:58.513944 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:58.515166 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:58.517106 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:58.517723 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:58.517849 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:58.519119 (+ 1270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:58.519126 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:58.519138 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:07:58.519632 (+ 494us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:58.519779 (+ 147us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:07:58.519803 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:58.519931 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:58.520289 (+ 358us) server_negotiation.cc:299] Negotiation successful
0811 12:07:58.520471 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":628,"thread_start_us":406,"threads_started":1}
I20260811 12:07:58.523254 9727 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a7885d936e94855a6f15c00fdca5cc1: Neither blocks nor log segments found. Creating new log.
I20260811 12:07:58.524521 9727 log.cc:824] T 00000000000000000000000000000000 P 7a7885d936e94855a6f15c00fdca5cc1: Log is configured to *not* fsync() on all Append() calls
I20260811 12:07:58.525173 10102 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:07:58.527184 9727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a7885d936e94855a6f15c00fdca5cc1: No bootstrap required, opened a new log
I20260811 12:07:58.535161 9727 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a7885d936e94855a6f15c00fdca5cc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a7885d936e94855a6f15c00fdca5cc1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37589 } }
I20260811 12:07:58.535526 9727 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7a7885d936e94855a6f15c00fdca5cc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:07:58.535696 9727 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a7885d936e94855a6f15c00fdca5cc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a7885d936e94855a6f15c00fdca5cc1, State: Initialized, Role: FOLLOWER
I20260811 12:07:58.535869 9734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:07:58.529552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36017 (local address 127.0.0.1:37589)
0811 12:07:58.529735 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 12:07:58.529746 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:07:58.530154 (+ 408us) server_negotiation.cc:407] Connection header received
0811 12:07:58.530335 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:07:58.530340 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:07:58.530427 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:07:58.530528 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:07:58.531011 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:58.531885 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:58.533200 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:07:58.533702 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:07:58.533866 (+ 164us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:07:58.534948 (+ 1082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:07:58.534954 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:07:58.534958 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:07:58.534983 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:07:58.535044 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:07:58.535078 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:07:58.535205 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 12:07:58.535402 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 12:07:58.535451 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 12:07:58.536346 9727 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a7885d936e94855a6f15c00fdca5cc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a7885d936e94855a6f15c00fdca5cc1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37589 } }
I20260811 12:07:58.536595 9727 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a7885d936e94855a6f15c00fdca5cc1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:07:58.536757 9727 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a7885d936e94855a6f15c00fdca5cc1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:07:58.536924 9727 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a7885d936e94855a6f15c00fdca5cc1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:07:58.539113 9727 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a7885d936e94855a6f15c00fdca5cc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a7885d936e94855a6f15c00fdca5cc1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37589 } }
I20260811 12:07:58.539615 9727 leader_election.cc:302] T 00000000000000000000000000000000 P 7a7885d936e94855a6f15c00fdca5cc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a7885d936e94855a6f15c00fdca5cc1; no voters:
I20260811 12:07:58.540244 9727 leader_election.cc:288] T 00000000000000000000000000000000 P 7a7885d936e94855a6f15c00fdca5cc1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:07:58.540342 9745 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a7885d936e94855a6f15c00fdca5cc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:07:58.541045 9745 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a7885d936e94855a6f15c00fdca5cc1 [term 1 LEADER]: Becoming Leader. State: Replica: 7a7885d936e94855a6f15c00fdca5cc1, State: Running, Role: LEADER
I20260811 12:07:58.541621 9745 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a7885d936e94855a6f15c00fdca5cc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a7885d936e94855a6f15c00fdca5cc1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37589 } }
I20260811 12:07:58.542526 9727 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a7885d936e94855a6f15c00fdca5cc1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:07:58.544541 9746 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a7885d936e94855a6f15c00fdca5cc1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7a7885d936e94855a6f15c00fdca5cc1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a7885d936e94855a6f15c00fdca5cc1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37589 } } }
I20260811 12:07:58.544960 9746 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a7885d936e94855a6f15c00fdca5cc1 [sys.catalog]: This master's current role is: LEADER
I20260811 12:07:58.544519 9747 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a7885d936e94855a6f15c00fdca5cc1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a7885d936e94855a6f15c00fdca5cc1. Latest consensus state: current_term: 1 leader_uuid: "7a7885d936e94855a6f15c00fdca5cc1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a7885d936e94855a6f15c00fdca5cc1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37589 } } }
I20260811 12:07:58.545814 9747 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a7885d936e94855a6f15c00fdca5cc1 [sys.catalog]: This master's current role is: LEADER
I20260811 12:07:58.546391 9750 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:07:58.551298 9750 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:07:58.561776 9750 catalog_manager.cc:1347] Generated new cluster ID: ac9d0b65253d4b1c9a9d7335b9ced04c
I20260811 12:07:58.561887 9750 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:07:58.585672 9750 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:07:58.587460 9750 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:07:58.598179 9750 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7a7885d936e94855a6f15c00fdca5cc1: Generated new TSK 0
I20260811 12:07:58.599169 9750 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:07:58.609874 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 9616
W20260811 12:07:58.610992 9712 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:34789
W20260811 12:07:58.611816 10102 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 9616: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:07:58.612004 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 9616
W20260811 12:07:58.639438 9597 connection.cc:535] client connection to 127.0.0.1:37589 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37589): Connection reset by peer (error 104)
2026-08-11T12:07:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2212 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449743084027-10102-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:07:58 dist-test-slave-mjjr krb5kdc[9771](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:07:58 dist-test-slave-mjjr krb5kdc[9771](info): set up 2 sockets
Aug 11 12:07:58 dist-test-slave-mjjr krb5kdc[9771](info): commencing operation
krb5kdc: starting...
W20260811 12:07:59.960341 10102 mini_kdc.cc:121] Time spent starting KDC: real 1.302s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Aug 11 12:08:00 dist-test-slave-mjjr krb5kdc[9771](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450080, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:08:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:08:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449743084027-10102-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449743084027-10102-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 12:08:00.264936 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36987
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449743084027-10102-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36987
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:08:00.606426 9795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:08:00.606729 9795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:08:00.606828 9795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:08:00.614001 9795 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:08:00.614111 9795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:08:00.614159 9795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:08:00.614223 9795 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:08:00.614266 9795 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:08:00.624686 9795 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36987
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449743084027-10102-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:08:00.626017 9795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:08:00.627362 9795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:08:00.639788 9802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:00.639772 9800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:00.639772 9805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:08:00.640872 9795 server_base.cc:1034] running on GCE node
I20260811 12:08:00.641631 9795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:08:00.642997 9795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:08:00.644249 9795 hybrid_clock.cc:648] HybridClock initialized: now 1786450080644195 us; error 45 us; skew 500 ppm
I20260811 12:08:00.644594 9795 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:08:00 dist-test-slave-mjjr krb5kdc[9771](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450080, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:08:00.647928 9795 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 12:08:00.649369 9795 webserver.cc:460] Webserver started at http://127.0.0.1:44047/ using document root <none> and password file <none>
I20260811 12:08:00.650051 9795 fs_manager.cc:359] Metadata directory not provided
I20260811 12:08:00.650198 9795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:08:00.650507 9795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:08:00.653467 9795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "8bac4742efe14b76a45ec251b41f17db"
format_stamp: "Formatted at 2026-08-11 12:08:00 on dist-test-slave-mjjr"
server_key: "053507b6f9853dc6e910585daec24f79"
server_key_iv: "6d6f5de494c09fac99c053e20faf90b1"
server_key_version: "encryptionkey@0"
I20260811 12:08:00.654223 9795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "8bac4742efe14b76a45ec251b41f17db"
format_stamp: "Formatted at 2026-08-11 12:08:00 on dist-test-slave-mjjr"
server_key: "053507b6f9853dc6e910585daec24f79"
server_key_iv: "6d6f5de494c09fac99c053e20faf90b1"
server_key_version: "encryptionkey@0"
I20260811 12:08:00.659225 9795 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:08:00.662472 9821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:08:00.663784 9795 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:08:00.663962 9795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "8bac4742efe14b76a45ec251b41f17db"
format_stamp: "Formatted at 2026-08-11 12:08:00 on dist-test-slave-mjjr"
server_key: "053507b6f9853dc6e910585daec24f79"
server_key_iv: "6d6f5de494c09fac99c053e20faf90b1"
server_key_version: "encryptionkey@0"
I20260811 12:08:00.664153 9795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:08:00.669605 9795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:08:00.673660 9795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:08:00.673974 9795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:08:00.692871 9795 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36987
I20260811 12:08:00.692957 9901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36987 every 8 connection(s)
I20260811 12:08:00.694262 9795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:08:00.697921 9902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:08:00.702283 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 9795
I20260811 12:08:00.702488 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:08:00.702941 10102 external_mini_cluster.cc:1428] Setting key 2f1f2d9cd3af17ecc33a727784e86553
I20260811 12:08:00.710376 9902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bac4742efe14b76a45ec251b41f17db: Bootstrap starting.
I20260811 12:08:00.712309 9906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:00.704797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37767 (local address 127.0.0.1:36987)
0811 12:08:00.705643 (+ 846us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:00.705658 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:00.705706 (+ 48us) server_negotiation.cc:407] Connection header received
0811 12:08:00.708504 (+ 2798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:00.708550 (+ 46us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:00.708966 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:00.709399 (+ 433us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:00.710379 (+ 980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:00.710387 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:00.710404 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:08:00.710422 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:00.710494 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:08:00.710529 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:00.710750 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:00.710881 (+ 131us) server_negotiation.cc:299] Negotiation successful
0811 12:08:00.711062 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":331,"threads_started":1}
I20260811 12:08:00.713933 9902 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8bac4742efe14b76a45ec251b41f17db: Neither blocks nor log segments found. Creating new log.
I20260811 12:08:00.715178 9902 log.cc:824] T 00000000000000000000000000000000 P 8bac4742efe14b76a45ec251b41f17db: Log is configured to *not* fsync() on all Append() calls
I20260811 12:08:00.715857 10102 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:08:00.717921 9902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bac4742efe14b76a45ec251b41f17db: No bootstrap required, opened a new log
I20260811 12:08:00.722525 9906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:00.720277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52769 (local address 127.0.0.1:36987)
0811 12:08:00.720436 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:00.720441 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:00.720770 (+ 329us) server_negotiation.cc:407] Connection header received
0811 12:08:00.720893 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:00.720898 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:00.720955 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:00.721058 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:00.721911 (+ 853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:00.721917 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:00.721920 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:08:00.721938 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:00.721982 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:08:00.722000 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:00.722106 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:00.722194 (+ 88us) server_negotiation.cc:299] Negotiation successful
0811 12:08:00.722251 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 12:08:00.726497 9902 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8bac4742efe14b76a45ec251b41f17db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bac4742efe14b76a45ec251b41f17db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36987 } }
I20260811 12:08:00.726948 9902 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8bac4742efe14b76a45ec251b41f17db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:08:00.727046 9902 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8bac4742efe14b76a45ec251b41f17db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bac4742efe14b76a45ec251b41f17db, State: Initialized, Role: FOLLOWER
I20260811 12:08:00.727617 9902 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8bac4742efe14b76a45ec251b41f17db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bac4742efe14b76a45ec251b41f17db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36987 } }
I20260811 12:08:00.727918 9902 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8bac4742efe14b76a45ec251b41f17db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:08:00.728084 9902 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8bac4742efe14b76a45ec251b41f17db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:08:00.728263 9902 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8bac4742efe14b76a45ec251b41f17db [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:08:00.730417 9902 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8bac4742efe14b76a45ec251b41f17db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bac4742efe14b76a45ec251b41f17db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36987 } }
I20260811 12:08:00.730928 9902 leader_election.cc:302] T 00000000000000000000000000000000 P 8bac4742efe14b76a45ec251b41f17db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8bac4742efe14b76a45ec251b41f17db; no voters:
I20260811 12:08:00.731504 9902 leader_election.cc:288] T 00000000000000000000000000000000 P 8bac4742efe14b76a45ec251b41f17db [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:08:00.731662 9914 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8bac4742efe14b76a45ec251b41f17db [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:08:00.732601 9914 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8bac4742efe14b76a45ec251b41f17db [term 1 LEADER]: Becoming Leader. State: Replica: 8bac4742efe14b76a45ec251b41f17db, State: Running, Role: LEADER
I20260811 12:08:00.733248 9914 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8bac4742efe14b76a45ec251b41f17db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bac4742efe14b76a45ec251b41f17db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36987 } }
I20260811 12:08:00.733773 9902 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8bac4742efe14b76a45ec251b41f17db [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:08:00.735873 9915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bac4742efe14b76a45ec251b41f17db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8bac4742efe14b76a45ec251b41f17db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bac4742efe14b76a45ec251b41f17db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36987 } } }
I20260811 12:08:00.736344 9915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bac4742efe14b76a45ec251b41f17db [sys.catalog]: This master's current role is: LEADER
I20260811 12:08:00.736932 9916 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bac4742efe14b76a45ec251b41f17db [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8bac4742efe14b76a45ec251b41f17db. Latest consensus state: current_term: 1 leader_uuid: "8bac4742efe14b76a45ec251b41f17db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bac4742efe14b76a45ec251b41f17db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36987 } } }
I20260811 12:08:00.737849 9916 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bac4742efe14b76a45ec251b41f17db [sys.catalog]: This master's current role is: LEADER
I20260811 12:08:00.744366 9932 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:08:00.749516 9932 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:08:00.759872 9932 catalog_manager.cc:1347] Generated new cluster ID: 80ce13c34fc941d9bd7cb5db312021ee
I20260811 12:08:00.759985 9932 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:08:00.781244 9932 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:08:00.781716 9932 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:08:00.789209 9932 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8bac4742efe14b76a45ec251b41f17db: Generated new TSK 0
I20260811 12:08:00.790030 9932 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:08:00.801585 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 9795
W20260811 12:08:00.802649 9891 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:37767
W20260811 12:08:00.803557 10102 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 9795: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:08:00.803779 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 9795
2026-08-11T12:08:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2187 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449743084027-10102-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:08:00 dist-test-slave-mjjr krb5kdc[9953](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:08:00 dist-test-slave-mjjr krb5kdc[9953](info): set up 2 sockets
Aug 11 12:08:00 dist-test-slave-mjjr krb5kdc[9953](info): commencing operation
krb5kdc: starting...
W20260811 12:08:02.145483 10102 mini_kdc.cc:121] Time spent starting KDC: real 1.299s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Aug 11 12:08:02 dist-test-slave-mjjr krb5kdc[9953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:08:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:08:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449743084027-10102-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449743084027-10102-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 12:08:02.447337 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36053
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449743084027-10102-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36053
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:08:02.788051 9974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:08:02.788403 9974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:08:02.788576 9974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:08:02.796483 9974 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:08:02.796660 9974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:08:02.796780 9974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:08:02.796898 9974 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:08:02.796983 9974 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:08:02.807192 9974 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36053
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449743084027-10102-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:08:02.808561 9974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:08:02.809950 9974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:08:02.822042 9985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:02.822158 9986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:02.822173 9988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:08:02.823321 9974 server_base.cc:1034] running on GCE node
I20260811 12:08:02.824160 9974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:08:02.825642 9974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:08:02.826886 9974 hybrid_clock.cc:648] HybridClock initialized: now 1786450082826836 us; error 49 us; skew 500 ppm
I20260811 12:08:02.827234 9974 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:08:02 dist-test-slave-mjjr krb5kdc[9953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450082, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:08:02.834800 9974 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 12:08:02.836338 9974 webserver.cc:460] Webserver started at http://127.0.0.1:46587/ using document root <none> and password file <none>
I20260811 12:08:02.836997 9974 fs_manager.cc:359] Metadata directory not provided
I20260811 12:08:02.837149 9974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:08:02.837464 9974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:08:02.840484 9974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "29f2413d374b42de9d6cba2287255b6a"
format_stamp: "Formatted at 2026-08-11 12:08:02 on dist-test-slave-mjjr"
server_key: "dd16744f1876d3807c7413d4b6d4cd84"
server_key_iv: "48456d9b86b112dd14447e49cbc224fd"
server_key_version: "encryptionkey@0"
I20260811 12:08:02.841238 9974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "29f2413d374b42de9d6cba2287255b6a"
format_stamp: "Formatted at 2026-08-11 12:08:02 on dist-test-slave-mjjr"
server_key: "dd16744f1876d3807c7413d4b6d4cd84"
server_key_iv: "48456d9b86b112dd14447e49cbc224fd"
server_key_version: "encryptionkey@0"
I20260811 12:08:02.846170 9974 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:08:02.849442 9999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:08:02.850613 9974 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 12:08:02.850806 9974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "29f2413d374b42de9d6cba2287255b6a"
format_stamp: "Formatted at 2026-08-11 12:08:02 on dist-test-slave-mjjr"
server_key: "dd16744f1876d3807c7413d4b6d4cd84"
server_key_iv: "48456d9b86b112dd14447e49cbc224fd"
server_key_version: "encryptionkey@0"
I20260811 12:08:02.851002 9974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:08:02.886142 9974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:08:02.890225 9974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:08:02.890535 9974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:08:02.909571 9974 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36053
I20260811 12:08:02.909615 10083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36053 every 8 connection(s)
I20260811 12:08:02.911005 9974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:08:02.914561 10085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:08:02.916270 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 9974
I20260811 12:08:02.916486 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:08:02.917030 10102 external_mini_cluster.cc:1428] Setting key f73c5e65325cf9aa565e39fe9cfee7ae
I20260811 12:08:02.930280 10085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29f2413d374b42de9d6cba2287255b6a: Bootstrap starting.
Aug 11 12:08:02 dist-test-slave-mjjr krb5kdc[9953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 12:08:02.934136 10085 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 29f2413d374b42de9d6cba2287255b6a: Neither blocks nor log segments found. Creating new log.
I20260811 12:08:02.935556 10085 log.cc:824] T 00000000000000000000000000000000 P 29f2413d374b42de9d6cba2287255b6a: Log is configured to *not* fsync() on all Append() calls
I20260811 12:08:02.938740 10085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29f2413d374b42de9d6cba2287255b6a: No bootstrap required, opened a new log
I20260811 12:08:02.941041 10088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:02.918797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54383 (local address 127.0.0.1:36053)
0811 12:08:02.919677 (+ 880us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:02.919693 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:02.919743 (+ 50us) server_negotiation.cc:407] Connection header received
0811 12:08:02.921348 (+ 1605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:02.921368 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:02.921729 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:02.922101 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:02.924185 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:02.925470 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:02.927570 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:02.928293 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:02.928426 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:02.933540 (+ 5114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:02.933547 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:02.933561 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:02.933600 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:02.936921 (+ 3321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:02.937702 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:02.937709 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:02.937715 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:02.937893 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:02.938301 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:02.938307 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:02.938311 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:02.938790 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:02.939098 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:02.939586 (+ 488us) server_negotiation.cc:299] Negotiation successful
0811 12:08:02.940058 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":416,"threads_started":1}
I20260811 12:08:02.944819 10102 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:08:02.948052 10085 raft_consensus.cc:348] T 00000000000000000000000000000000 P 29f2413d374b42de9d6cba2287255b6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29f2413d374b42de9d6cba2287255b6a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36053 } }
I20260811 12:08:02.948455 10085 raft_consensus.cc:374] T 00000000000000000000000000000000 P 29f2413d374b42de9d6cba2287255b6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:08:02.948590 10085 raft_consensus.cc:729] T 00000000000000000000000000000000 P 29f2413d374b42de9d6cba2287255b6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29f2413d374b42de9d6cba2287255b6a, State: Initialized, Role: FOLLOWER
I20260811 12:08:02.949193 10085 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29f2413d374b42de9d6cba2287255b6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29f2413d374b42de9d6cba2287255b6a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36053 } }
I20260811 12:08:02.949540 10085 raft_consensus.cc:388] T 00000000000000000000000000000000 P 29f2413d374b42de9d6cba2287255b6a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:08:02.949749 10085 raft_consensus.cc:482] T 00000000000000000000000000000000 P 29f2413d374b42de9d6cba2287255b6a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:08:02.950004 10085 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 29f2413d374b42de9d6cba2287255b6a [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:08:02.953599 10085 raft_consensus.cc:504] T 00000000000000000000000000000000 P 29f2413d374b42de9d6cba2287255b6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29f2413d374b42de9d6cba2287255b6a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36053 } }
I20260811 12:08:02.954272 10085 leader_election.cc:302] T 00000000000000000000000000000000 P 29f2413d374b42de9d6cba2287255b6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 29f2413d374b42de9d6cba2287255b6a; no voters:
I20260811 12:08:02.955003 10085 leader_election.cc:288] T 00000000000000000000000000000000 P 29f2413d374b42de9d6cba2287255b6a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:08:02.955130 10095 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 29f2413d374b42de9d6cba2287255b6a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:08:02.957551 10085 sys_catalog.cc:564] T 00000000000000000000000000000000 P 29f2413d374b42de9d6cba2287255b6a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:08:02.962189 10095 raft_consensus.cc:686] T 00000000000000000000000000000000 P 29f2413d374b42de9d6cba2287255b6a [term 1 LEADER]: Becoming Leader. State: Replica: 29f2413d374b42de9d6cba2287255b6a, State: Running, Role: LEADER
I20260811 12:08:02.962795 10095 consensus_queue.cc:237] T 00000000000000000000000000000000 P 29f2413d374b42de9d6cba2287255b6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29f2413d374b42de9d6cba2287255b6a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36053 } }
I20260811 12:08:02.966030 10097 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29f2413d374b42de9d6cba2287255b6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "29f2413d374b42de9d6cba2287255b6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29f2413d374b42de9d6cba2287255b6a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36053 } } }
I20260811 12:08:02.966527 10097 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29f2413d374b42de9d6cba2287255b6a [sys.catalog]: This master's current role is: LEADER
I20260811 12:08:02.969516 10088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:02.949618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39693 (local address 127.0.0.1:36053)
0811 12:08:02.949843 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:02.949850 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:02.950246 (+ 396us) server_negotiation.cc:407] Connection header received
0811 12:08:02.950451 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:02.950455 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:02.950515 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:02.950594 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:02.952733 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:02.953443 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:02.955448 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:02.956279 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:02.956387 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:02.960399 (+ 4012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:02.960409 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:02.960416 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:02.960466 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:02.964773 (+ 4307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:02.965785 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:02.965793 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:02.965798 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:02.965980 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:02.966513 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:02.966520 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:02.966525 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:02.967025 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:02.967360 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:02.968697 (+ 1337us) server_negotiation.cc:299] Negotiation successful
0811 12:08:02.968978 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 12:08:02.971174 10113 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:08:02.970968 10099 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29f2413d374b42de9d6cba2287255b6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 29f2413d374b42de9d6cba2287255b6a. Latest consensus state: current_term: 1 leader_uuid: "29f2413d374b42de9d6cba2287255b6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29f2413d374b42de9d6cba2287255b6a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36053 } } }
I20260811 12:08:02.971604 10099 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29f2413d374b42de9d6cba2287255b6a [sys.catalog]: This master's current role is: LEADER
I20260811 12:08:02.979427 10113 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:08:02.989915 10113 catalog_manager.cc:1347] Generated new cluster ID: 78e2f933cde44a08948936b09b3d1fc0
I20260811 12:08:02.990026 10113 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:08:02.997743 10113 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:08:02.999509 10113 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:08:03.011364 10113 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 29f2413d374b42de9d6cba2287255b6a: Generated new TSK 0
I20260811 12:08:03.012187 10113 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:08:03.019990 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 9974
I20260811 12:08:03.021072 10079 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:08:03.110057 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 9974
2026-08-11T12:08:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2309 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449743084027-10102-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:08:03 dist-test-slave-mjjr krb5kdc[10135](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:08:03 dist-test-slave-mjjr krb5kdc[10135](info): set up 2 sockets
Aug 11 12:08:03 dist-test-slave-mjjr krb5kdc[10135](info): commencing operation
krb5kdc: starting...
W20260811 12:08:04.431900 10102 mini_kdc.cc:121] Time spent starting KDC: real 1.276s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Aug 11 12:08:04 dist-test-slave-mjjr krb5kdc[10135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450084, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:08:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:08:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449743084027-10102-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449743084027-10102-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 12:08:04.724480 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37875
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449743084027-10102-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37875
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:08:05.062711 10157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:08:05.063063 10157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:08:05.063264 10157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:08:05.070432 10157 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:08:05.070575 10157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:08:05.070701 10157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:08:05.070834 10157 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:08:05.070911 10157 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:08:05.080495 10157 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37875
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449743084027-10102-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:08:05.081698 10157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:08:05.082948 10157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:08:05.095937 10167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:05.095888 10168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:05.095983 10171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:08:05.096680 10157 server_base.cc:1034] running on GCE node
I20260811 12:08:05.097362 10157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:08:05.098577 10157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:08:05.099745 10157 hybrid_clock.cc:648] HybridClock initialized: now 1786450085099691 us; error 62 us; skew 500 ppm
I20260811 12:08:05.100111 10157 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:08:05 dist-test-slave-mjjr krb5kdc[10135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450085, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:08:05.103176 10157 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 12:08:05.104595 10157 webserver.cc:460] Webserver started at http://127.0.0.1:43087/ using document root <none> and password file <none>
I20260811 12:08:05.105228 10157 fs_manager.cc:359] Metadata directory not provided
I20260811 12:08:05.105388 10157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:08:05.105684 10157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:08:05.108628 10157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "9455d8dfae854b02aafae5a1ba4a94a6"
format_stamp: "Formatted at 2026-08-11 12:08:05 on dist-test-slave-mjjr"
server_key: "eed765af7c76716327278f9828c256e6"
server_key_iv: "8638d6dbbc11c955be735fa02e948ef1"
server_key_version: "encryptionkey@0"
I20260811 12:08:05.109418 10157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "9455d8dfae854b02aafae5a1ba4a94a6"
format_stamp: "Formatted at 2026-08-11 12:08:05 on dist-test-slave-mjjr"
server_key: "eed765af7c76716327278f9828c256e6"
server_key_iv: "8638d6dbbc11c955be735fa02e948ef1"
server_key_version: "encryptionkey@0"
I20260811 12:08:05.114430 10157 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 12:08:05.117751 10183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:08:05.118908 10157 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 12:08:05.119094 10157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "9455d8dfae854b02aafae5a1ba4a94a6"
format_stamp: "Formatted at 2026-08-11 12:08:05 on dist-test-slave-mjjr"
server_key: "eed765af7c76716327278f9828c256e6"
server_key_iv: "8638d6dbbc11c955be735fa02e948ef1"
server_key_version: "encryptionkey@0"
I20260811 12:08:05.119282 10157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:08:05.144579 10157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:08:05.148521 10157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:08:05.148811 10157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:08:05.167447 10157 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37875
I20260811 12:08:05.167483 10266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37875 every 8 connection(s)
I20260811 12:08:05.168850 10157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:08:05.170452 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 10157
I20260811 12:08:05.170617 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:08:05.171084 10102 external_mini_cluster.cc:1428] Setting key c4fd4f85565c5b490d0da5b202e87ccc
I20260811 12:08:05.173079 10267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:08:05.184234 10271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:05.172880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58417 (local address 127.0.0.1:37875)
0811 12:08:05.173716 (+ 836us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:05.173728 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:05.173768 (+ 40us) server_negotiation.cc:407] Connection header received
0811 12:08:05.175338 (+ 1570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:05.175357 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:05.175809 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:05.176114 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:05.176788 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:05.178102 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:05.179963 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:05.180589 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:05.180714 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:05.181981 (+ 1267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:05.181992 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:05.182003 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:08:05.182546 (+ 543us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:05.182617 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:08:05.182640 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:05.182727 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:05.182976 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 12:08:05.183243 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":387,"threads_started":1}
I20260811 12:08:05.185724 10267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9455d8dfae854b02aafae5a1ba4a94a6: Bootstrap starting.
I20260811 12:08:05.187866 10102 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:08:05.188962 10267 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9455d8dfae854b02aafae5a1ba4a94a6: Neither blocks nor log segments found. Creating new log.
I20260811 12:08:05.189899 10267 log.cc:824] T 00000000000000000000000000000000 P 9455d8dfae854b02aafae5a1ba4a94a6: Log is configured to *not* fsync() on all Append() calls
I20260811 12:08:05.193181 10267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9455d8dfae854b02aafae5a1ba4a94a6: No bootstrap required, opened a new log
I20260811 12:08:05.198716 10271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:05.192344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32807 (local address 127.0.0.1:37875)
0811 12:08:05.192539 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:05.192546 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:05.192907 (+ 361us) server_negotiation.cc:407] Connection header received
0811 12:08:05.193106 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:05.193109 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:05.193161 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:05.193237 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:05.193854 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:05.194550 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:05.196405 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:05.196844 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:05.196938 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:05.198022 (+ 1084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:05.198029 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:05.198032 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:08:05.198051 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:05.198083 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:08:05.198099 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:05.198168 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:05.198313 (+ 145us) server_negotiation.cc:299] Negotiation successful
0811 12:08:05.198387 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 12:08:05.201902 10267 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9455d8dfae854b02aafae5a1ba4a94a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9455d8dfae854b02aafae5a1ba4a94a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37875 } }
I20260811 12:08:05.202239 10267 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9455d8dfae854b02aafae5a1ba4a94a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:08:05.202358 10267 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9455d8dfae854b02aafae5a1ba4a94a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9455d8dfae854b02aafae5a1ba4a94a6, State: Initialized, Role: FOLLOWER
I20260811 12:08:05.202986 10267 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9455d8dfae854b02aafae5a1ba4a94a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9455d8dfae854b02aafae5a1ba4a94a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37875 } }
I20260811 12:08:05.203269 10267 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9455d8dfae854b02aafae5a1ba4a94a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:08:05.203426 10267 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9455d8dfae854b02aafae5a1ba4a94a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:08:05.203624 10267 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9455d8dfae854b02aafae5a1ba4a94a6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:08:05.205762 10267 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9455d8dfae854b02aafae5a1ba4a94a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9455d8dfae854b02aafae5a1ba4a94a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37875 } }
I20260811 12:08:05.206269 10267 leader_election.cc:302] T 00000000000000000000000000000000 P 9455d8dfae854b02aafae5a1ba4a94a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9455d8dfae854b02aafae5a1ba4a94a6; no voters:
I20260811 12:08:05.206779 10267 leader_election.cc:288] T 00000000000000000000000000000000 P 9455d8dfae854b02aafae5a1ba4a94a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:08:05.206919 10282 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9455d8dfae854b02aafae5a1ba4a94a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:08:05.207621 10282 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9455d8dfae854b02aafae5a1ba4a94a6 [term 1 LEADER]: Becoming Leader. State: Replica: 9455d8dfae854b02aafae5a1ba4a94a6, State: Running, Role: LEADER
I20260811 12:08:05.208177 10282 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9455d8dfae854b02aafae5a1ba4a94a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9455d8dfae854b02aafae5a1ba4a94a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37875 } }
I20260811 12:08:05.208815 10267 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9455d8dfae854b02aafae5a1ba4a94a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:08:05.211097 10283 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9455d8dfae854b02aafae5a1ba4a94a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9455d8dfae854b02aafae5a1ba4a94a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9455d8dfae854b02aafae5a1ba4a94a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37875 } } }
I20260811 12:08:05.211036 10284 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9455d8dfae854b02aafae5a1ba4a94a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9455d8dfae854b02aafae5a1ba4a94a6. Latest consensus state: current_term: 1 leader_uuid: "9455d8dfae854b02aafae5a1ba4a94a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9455d8dfae854b02aafae5a1ba4a94a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37875 } } }
I20260811 12:08:05.211498 10283 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9455d8dfae854b02aafae5a1ba4a94a6 [sys.catalog]: This master's current role is: LEADER
I20260811 12:08:05.211529 10284 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9455d8dfae854b02aafae5a1ba4a94a6 [sys.catalog]: This master's current role is: LEADER
W20260811 12:08:05.219812 10302 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9455d8dfae854b02aafae5a1ba4a94a6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 12:08:05.219947 10302 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 12:08:05.220119 10292 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:08:05.221690 10292 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:08:05.229517 10292 catalog_manager.cc:1347] Generated new cluster ID: d5300ccdd95a4cc0b1b7e90c49dbc9d9
I20260811 12:08:05.229622 10292 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:08:05.237851 10292 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:08:05.239003 10292 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:08:05.254745 10292 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9455d8dfae854b02aafae5a1ba4a94a6: Generated new TSK 0
I20260811 12:08:05.255667 10292 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:08:05.262809 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 10157
W20260811 12:08:05.264086 10260 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:58417
W20260811 12:08:05.264757 10102 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 10157: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:08:05.264907 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 10157
2026-08-11T12:08:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2154 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449743084027-10102-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:08:05 dist-test-slave-mjjr krb5kdc[10311](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:08:05 dist-test-slave-mjjr krb5kdc[10311](info): set up 2 sockets
Aug 11 12:08:05 dist-test-slave-mjjr krb5kdc[10311](info): commencing operation
krb5kdc: starting...
W20260811 12:08:06.641724 10102 mini_kdc.cc:121] Time spent starting KDC: real 1.330s user 0.008s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Aug 11 12:08:06 dist-test-slave-mjjr krb5kdc[10311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450086, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:08:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:08:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449743084027-10102-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449743084027-10102-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 12:08:06.943507 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38259
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449743084027-10102-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38259
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:08:07.286696 10333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:08:07.287094 10333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:08:07.287263 10333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:08:07.294865 10333 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:08:07.295089 10333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:08:07.295217 10333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:08:07.295332 10333 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:08:07.295413 10333 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:08:07.306571 10333 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38259
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449743084027-10102-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:08:07.308174 10333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:08:07.309705 10333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:08:07.322480 10345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:07.322518 10341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:07.322708 10340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:08:07.323628 10333 server_base.cc:1034] running on GCE node
I20260811 12:08:07.324462 10333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:08:07.325932 10333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:08:07.327198 10333 hybrid_clock.cc:648] HybridClock initialized: now 1786450087327150 us; error 64 us; skew 500 ppm
I20260811 12:08:07.327564 10333 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:08:07 dist-test-slave-mjjr krb5kdc[10311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450087, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:08:07.330847 10333 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 12:08:07.332321 10333 webserver.cc:460] Webserver started at http://127.0.0.1:39021/ using document root <none> and password file <none>
I20260811 12:08:07.333062 10333 fs_manager.cc:359] Metadata directory not provided
I20260811 12:08:07.333230 10333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:08:07.333555 10333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:08:07.336647 10333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "beb933eaac2f4380865e8cf5da1d3b52"
format_stamp: "Formatted at 2026-08-11 12:08:07 on dist-test-slave-mjjr"
server_key: "51a82ed71449db70afc948ed86bb29d1"
server_key_iv: "e5beffbe58f57535a552c109d500d58b"
server_key_version: "encryptionkey@0"
I20260811 12:08:07.337453 10333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "beb933eaac2f4380865e8cf5da1d3b52"
format_stamp: "Formatted at 2026-08-11 12:08:07 on dist-test-slave-mjjr"
server_key: "51a82ed71449db70afc948ed86bb29d1"
server_key_iv: "e5beffbe58f57535a552c109d500d58b"
server_key_version: "encryptionkey@0"
I20260811 12:08:07.342653 10333 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:08:07.346141 10359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:08:07.347483 10333 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 12:08:07.347721 10333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "beb933eaac2f4380865e8cf5da1d3b52"
format_stamp: "Formatted at 2026-08-11 12:08:07 on dist-test-slave-mjjr"
server_key: "51a82ed71449db70afc948ed86bb29d1"
server_key_iv: "e5beffbe58f57535a552c109d500d58b"
server_key_version: "encryptionkey@0"
I20260811 12:08:07.347934 10333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:08:07.353513 10333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:08:07.357743 10333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:08:07.358047 10333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:08:07.376814 10333 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38259
I20260811 12:08:07.376904 10437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38259 every 8 connection(s)
I20260811 12:08:07.378167 10333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:08:07.381829 10439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:08:07.384006 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 10333
I20260811 12:08:07.384194 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:08:07.384631 10102 external_mini_cluster.cc:1428] Setting key 7b8204fd3e63f15a85e362c7ac9103fb
I20260811 12:08:07.392184 10442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:07.386324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51333 (local address 127.0.0.1:38259)
0811 12:08:07.387167 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:07.387178 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:07.387220 (+ 42us) server_negotiation.cc:407] Connection header received
0811 12:08:07.388860 (+ 1640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:07.388887 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:07.389177 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:07.389479 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:07.390574 (+ 1095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:07.390580 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:07.390591 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:08:07.390612 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:07.390668 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:08:07.390688 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:07.390858 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:07.391029 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 12:08:07.391255 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":415,"threads_started":1}
I20260811 12:08:07.393409 10439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P beb933eaac2f4380865e8cf5da1d3b52: Bootstrap starting.
I20260811 12:08:07.395583 10102 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:08:07.396708 10439 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P beb933eaac2f4380865e8cf5da1d3b52: Neither blocks nor log segments found. Creating new log.
I20260811 12:08:07.397920 10439 log.cc:824] T 00000000000000000000000000000000 P beb933eaac2f4380865e8cf5da1d3b52: Log is configured to *not* fsync() on all Append() calls
I20260811 12:08:07.402529 10439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P beb933eaac2f4380865e8cf5da1d3b52: No bootstrap required, opened a new log
I20260811 12:08:07.405351 10442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:07.402653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52989 (local address 127.0.0.1:38259)
0811 12:08:07.402873 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:07.402879 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:07.403269 (+ 390us) server_negotiation.cc:407] Connection header received
0811 12:08:07.403479 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:07.403483 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:07.403530 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:07.403614 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:07.404658 (+ 1044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:07.404662 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:07.404665 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:08:07.404680 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:07.404718 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:08:07.404737 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:07.404835 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:07.404970 (+ 135us) server_negotiation.cc:299] Negotiation successful
0811 12:08:07.405018 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 12:08:07.411531 10439 raft_consensus.cc:348] T 00000000000000000000000000000000 P beb933eaac2f4380865e8cf5da1d3b52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb933eaac2f4380865e8cf5da1d3b52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38259 } }
I20260811 12:08:07.411984 10439 raft_consensus.cc:374] T 00000000000000000000000000000000 P beb933eaac2f4380865e8cf5da1d3b52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:08:07.412094 10439 raft_consensus.cc:729] T 00000000000000000000000000000000 P beb933eaac2f4380865e8cf5da1d3b52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: beb933eaac2f4380865e8cf5da1d3b52, State: Initialized, Role: FOLLOWER
I20260811 12:08:07.412768 10439 consensus_queue.cc:260] T 00000000000000000000000000000000 P beb933eaac2f4380865e8cf5da1d3b52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb933eaac2f4380865e8cf5da1d3b52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38259 } }
I20260811 12:08:07.413057 10439 raft_consensus.cc:388] T 00000000000000000000000000000000 P beb933eaac2f4380865e8cf5da1d3b52 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:08:07.413219 10439 raft_consensus.cc:482] T 00000000000000000000000000000000 P beb933eaac2f4380865e8cf5da1d3b52 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:08:07.413405 10439 raft_consensus.cc:3037] T 00000000000000000000000000000000 P beb933eaac2f4380865e8cf5da1d3b52 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:08:07.415632 10439 raft_consensus.cc:504] T 00000000000000000000000000000000 P beb933eaac2f4380865e8cf5da1d3b52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb933eaac2f4380865e8cf5da1d3b52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38259 } }
I20260811 12:08:07.416209 10439 leader_election.cc:302] T 00000000000000000000000000000000 P beb933eaac2f4380865e8cf5da1d3b52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: beb933eaac2f4380865e8cf5da1d3b52; no voters:
I20260811 12:08:07.416826 10439 leader_election.cc:288] T 00000000000000000000000000000000 P beb933eaac2f4380865e8cf5da1d3b52 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:08:07.416980 10460 raft_consensus.cc:2781] T 00000000000000000000000000000000 P beb933eaac2f4380865e8cf5da1d3b52 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:08:07.417652 10460 raft_consensus.cc:686] T 00000000000000000000000000000000 P beb933eaac2f4380865e8cf5da1d3b52 [term 1 LEADER]: Becoming Leader. State: Replica: beb933eaac2f4380865e8cf5da1d3b52, State: Running, Role: LEADER
I20260811 12:08:07.418190 10460 consensus_queue.cc:237] T 00000000000000000000000000000000 P beb933eaac2f4380865e8cf5da1d3b52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb933eaac2f4380865e8cf5da1d3b52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38259 } }
I20260811 12:08:07.418918 10439 sys_catalog.cc:564] T 00000000000000000000000000000000 P beb933eaac2f4380865e8cf5da1d3b52 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:08:07.421136 10464 sys_catalog.cc:455] T 00000000000000000000000000000000 P beb933eaac2f4380865e8cf5da1d3b52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader beb933eaac2f4380865e8cf5da1d3b52. Latest consensus state: current_term: 1 leader_uuid: "beb933eaac2f4380865e8cf5da1d3b52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb933eaac2f4380865e8cf5da1d3b52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38259 } } }
I20260811 12:08:07.421650 10464 sys_catalog.cc:458] T 00000000000000000000000000000000 P beb933eaac2f4380865e8cf5da1d3b52 [sys.catalog]: This master's current role is: LEADER
I20260811 12:08:07.421814 10462 sys_catalog.cc:455] T 00000000000000000000000000000000 P beb933eaac2f4380865e8cf5da1d3b52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "beb933eaac2f4380865e8cf5da1d3b52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb933eaac2f4380865e8cf5da1d3b52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38259 } } }
I20260811 12:08:07.422133 10462 sys_catalog.cc:458] T 00000000000000000000000000000000 P beb933eaac2f4380865e8cf5da1d3b52 [sys.catalog]: This master's current role is: LEADER
I20260811 12:08:07.424413 10474 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:08:07.429669 10474 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:08:07.438093 10474 catalog_manager.cc:1347] Generated new cluster ID: e7147e445f1b46538169592c1064fe8c
I20260811 12:08:07.438202 10474 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:08:07.450848 10474 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:08:07.451319 10474 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:08:07.457348 10474 catalog_manager.cc:5868] T 00000000000000000000000000000000 P beb933eaac2f4380865e8cf5da1d3b52: Generated new TSK 0
I20260811 12:08:07.458104 10474 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:08:07.471558 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 10333
W20260811 12:08:07.472688 10432 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:51333
W20260811 12:08:07.473454 10102 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 10333: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:08:07.473615 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 10333
2026-08-11T12:08:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2210 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449743084027-10102-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:08:07 dist-test-slave-mjjr krb5kdc[10492](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:08:07 dist-test-slave-mjjr krb5kdc[10492](info): set up 2 sockets
Aug 11 12:08:07 dist-test-slave-mjjr krb5kdc[10492](info): commencing operation
krb5kdc: starting...
W20260811 12:08:08.822093 10102 mini_kdc.cc:121] Time spent starting KDC: real 1.299s user 0.006s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Aug 11 12:08:09 dist-test-slave-mjjr krb5kdc[10492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:08:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:08:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449743084027-10102-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449743084027-10102-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 12:08:09.150559 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43629
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449743084027-10102-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43629
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:08:09.489401 10509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:08:09.489676 10509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:08:09.489840 10509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:08:09.497130 10509 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:08:09.497242 10509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:08:09.497335 10509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:08:09.497464 10509 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:08:09.497531 10509 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:08:09.507452 10509 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43629
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449743084027-10102-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:08:09.508778 10509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:08:09.510089 10509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:08:09.521914 10520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:09.521967 10521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:09.521971 10523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:08:09.523135 10509 server_base.cc:1034] running on GCE node
I20260811 12:08:09.523962 10509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:08:09.525254 10509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:08:09.526458 10509 hybrid_clock.cc:648] HybridClock initialized: now 1786450089526405 us; error 45 us; skew 500 ppm
I20260811 12:08:09.526799 10509 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:08:09 dist-test-slave-mjjr krb5kdc[10492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450089, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:08:09.530032 10509 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 12:08:09.531555 10509 webserver.cc:460] Webserver started at http://127.0.0.1:41731/ using document root <none> and password file <none>
I20260811 12:08:09.532236 10509 fs_manager.cc:359] Metadata directory not provided
I20260811 12:08:09.532389 10509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:08:09.532708 10509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:08:09.535756 10509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "77ffc49e82394fa28cd1e680fd0e3829"
format_stamp: "Formatted at 2026-08-11 12:08:09 on dist-test-slave-mjjr"
server_key: "9652dc21a6958df189a9540f5dde63e9"
server_key_iv: "a5cd1b3bd6da4669b3e90aa3aa5401ef"
server_key_version: "encryptionkey@0"
I20260811 12:08:09.536568 10509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "77ffc49e82394fa28cd1e680fd0e3829"
format_stamp: "Formatted at 2026-08-11 12:08:09 on dist-test-slave-mjjr"
server_key: "9652dc21a6958df189a9540f5dde63e9"
server_key_iv: "a5cd1b3bd6da4669b3e90aa3aa5401ef"
server_key_version: "encryptionkey@0"
I20260811 12:08:09.541554 10509 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260811 12:08:09.544920 10537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:08:09.546234 10509 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:08:09.546437 10509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "77ffc49e82394fa28cd1e680fd0e3829"
format_stamp: "Formatted at 2026-08-11 12:08:09 on dist-test-slave-mjjr"
server_key: "9652dc21a6958df189a9540f5dde63e9"
server_key_iv: "a5cd1b3bd6da4669b3e90aa3aa5401ef"
server_key_version: "encryptionkey@0"
I20260811 12:08:09.546640 10509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:08:09.575937 10509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:08:09.580412 10509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:08:09.580735 10509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:08:09.600137 10509 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43629
I20260811 12:08:09.600185 10630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43629 every 8 connection(s)
I20260811 12:08:09.601706 10509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:08:09.605644 10631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:08:09.612030 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 10509
I20260811 12:08:09.612202 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:08:09.612660 10102 external_mini_cluster.cc:1428] Setting key bc78f60b8cbfa7dba3837e2577f449c3
I20260811 12:08:09.617082 10631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77ffc49e82394fa28cd1e680fd0e3829: Bootstrap starting.
I20260811 12:08:09.620779 10631 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 77ffc49e82394fa28cd1e680fd0e3829: Neither blocks nor log segments found. Creating new log.
I20260811 12:08:09.622072 10631 log.cc:824] T 00000000000000000000000000000000 P 77ffc49e82394fa28cd1e680fd0e3829: Log is configured to *not* fsync() on all Append() calls
I20260811 12:08:09.625443 10631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77ffc49e82394fa28cd1e680fd0e3829: No bootstrap required, opened a new log
Aug 11 12:08:09 dist-test-slave-mjjr krb5kdc[10492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 12:08:09.633564 10631 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77ffc49e82394fa28cd1e680fd0e3829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77ffc49e82394fa28cd1e680fd0e3829" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43629 } }
I20260811 12:08:09.633948 10631 raft_consensus.cc:374] T 00000000000000000000000000000000 P 77ffc49e82394fa28cd1e680fd0e3829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:08:09.634053 10631 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77ffc49e82394fa28cd1e680fd0e3829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77ffc49e82394fa28cd1e680fd0e3829, State: Initialized, Role: FOLLOWER
I20260811 12:08:09.634640 10631 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77ffc49e82394fa28cd1e680fd0e3829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77ffc49e82394fa28cd1e680fd0e3829" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43629 } }
I20260811 12:08:09.634914 10631 raft_consensus.cc:388] T 00000000000000000000000000000000 P 77ffc49e82394fa28cd1e680fd0e3829 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:08:09.635077 10631 raft_consensus.cc:482] T 00000000000000000000000000000000 P 77ffc49e82394fa28cd1e680fd0e3829 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:08:09.635272 10631 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77ffc49e82394fa28cd1e680fd0e3829 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:08:09.637383 10637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:09.614413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.116:50723 (local address 127.0.0.1:43629)
0811 12:08:09.615286 (+ 873us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:09.615297 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:09.615346 (+ 49us) server_negotiation.cc:407] Connection header received
0811 12:08:09.617005 (+ 1659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:09.617023 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:09.617412 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:09.617696 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:09.619702 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:09.621003 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:09.622823 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:09.623823 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:09.623952 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:09.628980 (+ 5028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:09.628987 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:09.629000 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:09.629035 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:09.632294 (+ 3259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:09.633102 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:09.633108 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:09.633114 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:09.633282 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:09.633785 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:09.633789 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:09.633791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:09.634527 (+ 736us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:09.634872 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:09.635545 (+ 673us) server_negotiation.cc:299] Negotiation successful
0811 12:08:09.635936 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":663,"thread_start_us":413,"threads_started":1}
I20260811 12:08:09.638689 10631 raft_consensus.cc:504] T 00000000000000000000000000000000 P 77ffc49e82394fa28cd1e680fd0e3829 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77ffc49e82394fa28cd1e680fd0e3829" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43629 } }
I20260811 12:08:09.639258 10631 leader_election.cc:302] T 00000000000000000000000000000000 P 77ffc49e82394fa28cd1e680fd0e3829 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 77ffc49e82394fa28cd1e680fd0e3829; no voters:
I20260811 12:08:09.639945 10631 leader_election.cc:288] T 00000000000000000000000000000000 P 77ffc49e82394fa28cd1e680fd0e3829 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:08:09.640069 10644 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 77ffc49e82394fa28cd1e680fd0e3829 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:08:09.641963 10102 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:08:09.641932 10644 raft_consensus.cc:686] T 00000000000000000000000000000000 P 77ffc49e82394fa28cd1e680fd0e3829 [term 1 LEADER]: Becoming Leader. State: Replica: 77ffc49e82394fa28cd1e680fd0e3829, State: Running, Role: LEADER
I20260811 12:08:09.642170 10631 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77ffc49e82394fa28cd1e680fd0e3829 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:08:09.642446 10644 consensus_queue.cc:237] T 00000000000000000000000000000000 P 77ffc49e82394fa28cd1e680fd0e3829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77ffc49e82394fa28cd1e680fd0e3829" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43629 } }
I20260811 12:08:09.645895 10648 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77ffc49e82394fa28cd1e680fd0e3829 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 77ffc49e82394fa28cd1e680fd0e3829. Latest consensus state: current_term: 1 leader_uuid: "77ffc49e82394fa28cd1e680fd0e3829" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77ffc49e82394fa28cd1e680fd0e3829" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43629 } } }
I20260811 12:08:09.646356 10648 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77ffc49e82394fa28cd1e680fd0e3829 [sys.catalog]: This master's current role is: LEADER
I20260811 12:08:09.647059 10646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77ffc49e82394fa28cd1e680fd0e3829 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "77ffc49e82394fa28cd1e680fd0e3829" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77ffc49e82394fa28cd1e680fd0e3829" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43629 } } }
I20260811 12:08:09.647375 10646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77ffc49e82394fa28cd1e680fd0e3829 [sys.catalog]: This master's current role is: LEADER
I20260811 12:08:09.653362 10662 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:08:09.661474 10662 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:08:09.682796 10662 catalog_manager.cc:1347] Generated new cluster ID: d239e131f90a4ba3b7e0a3710ceffa9e
I20260811 12:08:09.682960 10662 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:08:09.684202 10637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:09.660974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.116:45471 (local address 127.0.0.1:43629)
0811 12:08:09.661149 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:09.661156 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:09.664323 (+ 3167us) server_negotiation.cc:407] Connection header received
0811 12:08:09.664634 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:09.664643 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:09.664757 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:09.664853 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:09.667603 (+ 2750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:09.669250 (+ 1647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:09.671839 (+ 2589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:09.672787 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:09.672920 (+ 133us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:09.677260 (+ 4340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:09.677265 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:09.677268 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:09.677297 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:09.680607 (+ 3310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:09.681598 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:09.681603 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:09.681617 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:09.681769 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:09.682293 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:09.682297 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:09.682300 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:09.682788 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:09.683103 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:09.683728 (+ 625us) server_negotiation.cc:299] Negotiation successful
0811 12:08:09.683921 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 12:08:09.702479 10662 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:08:09.703840 10662 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:08:09.710330 10662 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 77ffc49e82394fa28cd1e680fd0e3829: Generated new TSK 0
I20260811 12:08:09.711086 10662 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:08:09.720130 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 10509
I20260811 12:08:09.721616 10622 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:08:09.812983 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 10509
W20260811 12:08:09.836776 10489 connection.cc:535] client connection to 127.0.0.1:43629 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43629): Connection reset by peer (error 104)
2026-08-11T12:08:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2333 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449743084027-10102-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:08:09 dist-test-slave-mjjr krb5kdc[10694](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:08:09 dist-test-slave-mjjr krb5kdc[10694](info): set up 2 sockets
Aug 11 12:08:09 dist-test-slave-mjjr krb5kdc[10694](info): commencing operation
krb5kdc: starting...
W20260811 12:08:11.159710 10102 mini_kdc.cc:121] Time spent starting KDC: real 1.304s user 0.003s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Aug 11 12:08:11 dist-test-slave-mjjr krb5kdc[10694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450091, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:08:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:08:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449743084027-10102-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449743084027-10102-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 12:08:11.458634 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43367
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449743084027-10102-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43367
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:08:11.803663 10715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:08:11.804042 10715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:08:11.804134 10715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:08:11.811380 10715 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:08:11.811496 10715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:08:11.811553 10715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:08:11.811614 10715 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:08:11.811673 10715 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:08:11.821964 10715 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43367
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449743084027-10102-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:08:11.823292 10715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:08:11.824667 10715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:08:11.837182 10731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:11.837215 10728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:11.837469 10727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:08:11.837898 10715 server_base.cc:1034] running on GCE node
I20260811 12:08:11.838601 10715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:08:11.839920 10715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:08:11.841107 10715 hybrid_clock.cc:648] HybridClock initialized: now 1786450091841055 us; error 54 us; skew 500 ppm
I20260811 12:08:11.841441 10715 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:08:11 dist-test-slave-mjjr krb5kdc[10694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450091, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:08:11.844684 10715 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 12:08:11.846074 10715 webserver.cc:460] Webserver started at http://127.0.0.1:36633/ using document root <none> and password file <none>
I20260811 12:08:11.846779 10715 fs_manager.cc:359] Metadata directory not provided
I20260811 12:08:11.846925 10715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:08:11.847223 10715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:08:11.850160 10715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "e9c8a53dbf0141d3b928a6781cd4ed24"
format_stamp: "Formatted at 2026-08-11 12:08:11 on dist-test-slave-mjjr"
server_key: "897705f76bcd19413fea8da6d817ceb8"
server_key_iv: "4a938efbc00fb632e487adbff108591f"
server_key_version: "encryptionkey@0"
I20260811 12:08:11.850927 10715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "e9c8a53dbf0141d3b928a6781cd4ed24"
format_stamp: "Formatted at 2026-08-11 12:08:11 on dist-test-slave-mjjr"
server_key: "897705f76bcd19413fea8da6d817ceb8"
server_key_iv: "4a938efbc00fb632e487adbff108591f"
server_key_version: "encryptionkey@0"
I20260811 12:08:11.855963 10715 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260811 12:08:11.859190 10742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:08:11.860450 10715 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:08:11.860615 10715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "e9c8a53dbf0141d3b928a6781cd4ed24"
format_stamp: "Formatted at 2026-08-11 12:08:11 on dist-test-slave-mjjr"
server_key: "897705f76bcd19413fea8da6d817ceb8"
server_key_iv: "4a938efbc00fb632e487adbff108591f"
server_key_version: "encryptionkey@0"
I20260811 12:08:11.860819 10715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:08:11.880828 10715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:08:11.884552 10715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:08:11.884828 10715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:08:11.903291 10715 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43367
I20260811 12:08:11.903331 10822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43367 every 8 connection(s)
I20260811 12:08:11.904677 10715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:08:11.907786 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 10715
I20260811 12:08:11.907979 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:08:11.908452 10102 external_mini_cluster.cc:1428] Setting key a35d2fdd41e7336b15c0a78cf23de492
I20260811 12:08:11.908510 10823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:08:11.920775 10827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:11.910001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.116:46969 (local address 127.0.0.1:43367)
0811 12:08:11.910631 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:11.910642 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:11.910680 (+ 38us) server_negotiation.cc:407] Connection header received
0811 12:08:11.912349 (+ 1669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:11.912370 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:11.912743 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:11.913150 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:11.913693 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:11.915042 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:11.916423 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:11.917064 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:11.917263 (+ 199us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:11.918566 (+ 1303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:11.918573 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:11.918589 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:08:11.919167 (+ 578us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:11.919238 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:08:11.919261 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:11.919449 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:11.919731 (+ 282us) server_negotiation.cc:299] Negotiation successful
0811 12:08:11.919894 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":268,"threads_started":1}
I20260811 12:08:11.921222 10823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9c8a53dbf0141d3b928a6781cd4ed24: Bootstrap starting.
I20260811 12:08:11.924460 10102 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:08:11.924479 10823 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e9c8a53dbf0141d3b928a6781cd4ed24: Neither blocks nor log segments found. Creating new log.
I20260811 12:08:11.925467 10823 log.cc:824] T 00000000000000000000000000000000 P e9c8a53dbf0141d3b928a6781cd4ed24: Log is configured to *not* fsync() on all Append() calls
I20260811 12:08:11.928372 10823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9c8a53dbf0141d3b928a6781cd4ed24: No bootstrap required, opened a new log
I20260811 12:08:11.935217 10827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:11.928732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.116:46195 (local address 127.0.0.1:43367)
0811 12:08:11.928913 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:11.928920 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:11.929253 (+ 333us) server_negotiation.cc:407] Connection header received
0811 12:08:11.929403 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:11.929408 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:11.929479 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:11.929586 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:11.930032 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:11.931231 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:11.932574 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:11.933251 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:11.933399 (+ 148us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:11.934432 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:11.934438 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:11.934443 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:08:11.934463 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:11.934512 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:08:11.934535 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:11.934671 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:11.934821 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 12:08:11.934875 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 12:08:11.938006 10823 raft_consensus.cc:348] T 00000000000000000000000000000000 P e9c8a53dbf0141d3b928a6781cd4ed24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9c8a53dbf0141d3b928a6781cd4ed24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43367 } }
I20260811 12:08:11.938340 10823 raft_consensus.cc:374] T 00000000000000000000000000000000 P e9c8a53dbf0141d3b928a6781cd4ed24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:08:11.938416 10823 raft_consensus.cc:729] T 00000000000000000000000000000000 P e9c8a53dbf0141d3b928a6781cd4ed24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9c8a53dbf0141d3b928a6781cd4ed24, State: Initialized, Role: FOLLOWER
I20260811 12:08:11.938961 10823 consensus_queue.cc:260] T 00000000000000000000000000000000 P e9c8a53dbf0141d3b928a6781cd4ed24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9c8a53dbf0141d3b928a6781cd4ed24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43367 } }
I20260811 12:08:11.939168 10823 raft_consensus.cc:388] T 00000000000000000000000000000000 P e9c8a53dbf0141d3b928a6781cd4ed24 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:08:11.939266 10823 raft_consensus.cc:482] T 00000000000000000000000000000000 P e9c8a53dbf0141d3b928a6781cd4ed24 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:08:11.939391 10823 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e9c8a53dbf0141d3b928a6781cd4ed24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:08:11.941781 10823 raft_consensus.cc:504] T 00000000000000000000000000000000 P e9c8a53dbf0141d3b928a6781cd4ed24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9c8a53dbf0141d3b928a6781cd4ed24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43367 } }
I20260811 12:08:11.942232 10823 leader_election.cc:302] T 00000000000000000000000000000000 P e9c8a53dbf0141d3b928a6781cd4ed24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e9c8a53dbf0141d3b928a6781cd4ed24; no voters:
I20260811 12:08:11.942728 10823 leader_election.cc:288] T 00000000000000000000000000000000 P e9c8a53dbf0141d3b928a6781cd4ed24 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:08:11.942925 10838 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e9c8a53dbf0141d3b928a6781cd4ed24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:08:11.943699 10838 raft_consensus.cc:686] T 00000000000000000000000000000000 P e9c8a53dbf0141d3b928a6781cd4ed24 [term 1 LEADER]: Becoming Leader. State: Replica: e9c8a53dbf0141d3b928a6781cd4ed24, State: Running, Role: LEADER
I20260811 12:08:11.944350 10838 consensus_queue.cc:237] T 00000000000000000000000000000000 P e9c8a53dbf0141d3b928a6781cd4ed24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9c8a53dbf0141d3b928a6781cd4ed24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43367 } }
I20260811 12:08:11.945252 10823 sys_catalog.cc:564] T 00000000000000000000000000000000 P e9c8a53dbf0141d3b928a6781cd4ed24 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:08:11.947206 10841 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9c8a53dbf0141d3b928a6781cd4ed24 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9c8a53dbf0141d3b928a6781cd4ed24. Latest consensus state: current_term: 1 leader_uuid: "e9c8a53dbf0141d3b928a6781cd4ed24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9c8a53dbf0141d3b928a6781cd4ed24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43367 } } }
I20260811 12:08:11.947690 10841 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9c8a53dbf0141d3b928a6781cd4ed24 [sys.catalog]: This master's current role is: LEADER
I20260811 12:08:11.948856 10840 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9c8a53dbf0141d3b928a6781cd4ed24 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e9c8a53dbf0141d3b928a6781cd4ed24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9c8a53dbf0141d3b928a6781cd4ed24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43367 } } }
I20260811 12:08:11.949311 10840 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9c8a53dbf0141d3b928a6781cd4ed24 [sys.catalog]: This master's current role is: LEADER
I20260811 12:08:11.952756 10850 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:08:11.958554 10850 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:08:11.967662 10850 catalog_manager.cc:1347] Generated new cluster ID: ccf93cc1ca444f9ea07bcb3a043befcd
I20260811 12:08:11.967798 10850 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:08:11.984588 10850 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:08:11.985934 10850 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:08:11.990819 10850 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e9c8a53dbf0141d3b928a6781cd4ed24: Generated new TSK 0
I20260811 12:08:11.991616 10850 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:08:12.000890 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 10715
W20260811 12:08:12.002014 10814 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.116:46969
W20260811 12:08:12.002699 10102 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 10715: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:08:12.002851 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 10715
W20260811 12:08:12.028564 10692 connection.cc:535] client connection to 127.0.0.1:43367 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43367): Connection reset by peer (error 104)
2026-08-11T12:08:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2192 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449743084027-10102-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:08:12 dist-test-slave-mjjr krb5kdc[10876](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:08:12 dist-test-slave-mjjr krb5kdc[10876](info): set up 2 sockets
Aug 11 12:08:12 dist-test-slave-mjjr krb5kdc[10876](info): commencing operation
krb5kdc: starting...
W20260811 12:08:13.361485 10102 mini_kdc.cc:121] Time spent starting KDC: real 1.313s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Aug 11 12:08:13 dist-test-slave-mjjr krb5kdc[10876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:08:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:08:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449743084027-10102-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449743084027-10102-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 12:08:13.666275 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43117
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449743084027-10102-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43117
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:08:14.014484 10896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:08:14.014755 10896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:08:14.014951 10896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:08:14.021963 10896 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:08:14.022085 10896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:08:14.022172 10896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:08:14.022269 10896 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:08:14.022348 10896 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:08:14.031890 10896 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43117
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449743084027-10102-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:08:14.033169 10896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:08:14.034523 10896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:08:14.046438 10910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:14.046511 10913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:14.046532 10911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:08:14.047685 10896 server_base.cc:1034] running on GCE node
I20260811 12:08:14.048530 10896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:08:14.049945 10896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:08:14.051170 10896 hybrid_clock.cc:648] HybridClock initialized: now 1786450094051112 us; error 60 us; skew 500 ppm
I20260811 12:08:14.051510 10896 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:08:14 dist-test-slave-mjjr krb5kdc[10876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450094, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:08:14.054770 10896 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 12:08:14.056237 10896 webserver.cc:460] Webserver started at http://127.0.0.1:37285/ using document root <none> and password file <none>
I20260811 12:08:14.056913 10896 fs_manager.cc:359] Metadata directory not provided
I20260811 12:08:14.057056 10896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:08:14.057368 10896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:08:14.060393 10896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "eb0762c2b3794f39a1c948c4a497249c"
format_stamp: "Formatted at 2026-08-11 12:08:14 on dist-test-slave-mjjr"
server_key: "7c0878f7fb679c62e6a050c28789f7a3"
server_key_iv: "b01ab8dfc7fd47a448171ff25ddda854"
server_key_version: "encryptionkey@0"
I20260811 12:08:14.061153 10896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "eb0762c2b3794f39a1c948c4a497249c"
format_stamp: "Formatted at 2026-08-11 12:08:14 on dist-test-slave-mjjr"
server_key: "7c0878f7fb679c62e6a050c28789f7a3"
server_key_iv: "b01ab8dfc7fd47a448171ff25ddda854"
server_key_version: "encryptionkey@0"
I20260811 12:08:14.066380 10896 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:08:14.069811 10924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:08:14.071071 10896 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:08:14.071246 10896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "eb0762c2b3794f39a1c948c4a497249c"
format_stamp: "Formatted at 2026-08-11 12:08:14 on dist-test-slave-mjjr"
server_key: "7c0878f7fb679c62e6a050c28789f7a3"
server_key_iv: "b01ab8dfc7fd47a448171ff25ddda854"
server_key_version: "encryptionkey@0"
I20260811 12:08:14.071453 10896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:08:14.077126 10896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:08:14.085850 10896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:08:14.086174 10896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:08:14.105185 10896 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43117
I20260811 12:08:14.105261 10996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43117 every 8 connection(s)
I20260811 12:08:14.106607 10896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:08:14.110404 10997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:08:14.115106 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 10896
I20260811 12:08:14.115315 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:08:14.115947 10102 external_mini_cluster.cc:1428] Setting key 562252ddd14db648cc8a7ae8ada3dd89
I20260811 12:08:14.123947 10997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb0762c2b3794f39a1c948c4a497249c: Bootstrap starting.
I20260811 12:08:14.126225 11002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:14.117988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.116:42655 (local address 127.0.0.1:43117)
0811 12:08:14.119009 (+ 1021us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:14.119025 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:14.119352 (+ 327us) server_negotiation.cc:407] Connection header received
0811 12:08:14.122242 (+ 2890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:14.122270 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:14.122688 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:14.123128 (+ 440us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:14.124177 (+ 1049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:14.124185 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:14.124201 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:08:14.124220 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:14.124308 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:08:14.124346 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:14.124584 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:14.124727 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 12:08:14.124939 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":728,"thread_start_us":429,"threads_started":1}
I20260811 12:08:14.127985 10997 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb0762c2b3794f39a1c948c4a497249c: Neither blocks nor log segments found. Creating new log.
I20260811 12:08:14.129266 10997 log.cc:824] T 00000000000000000000000000000000 P eb0762c2b3794f39a1c948c4a497249c: Log is configured to *not* fsync() on all Append() calls
I20260811 12:08:14.129861 10102 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:08:14.132161 10997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb0762c2b3794f39a1c948c4a497249c: No bootstrap required, opened a new log
I20260811 12:08:14.137391 11002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:14.134674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.116:56649 (local address 127.0.0.1:43117)
0811 12:08:14.134849 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:14.134857 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:14.135404 (+ 547us) server_negotiation.cc:407] Connection header received
0811 12:08:14.135572 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:14.135575 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:14.135621 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:14.135749 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:14.136707 (+ 958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:14.136711 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:14.136714 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:08:14.136728 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:14.136763 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:08:14.136776 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:14.136865 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:14.137021 (+ 156us) server_negotiation.cc:299] Negotiation successful
0811 12:08:14.137067 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 12:08:14.142032 10997 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb0762c2b3794f39a1c948c4a497249c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb0762c2b3794f39a1c948c4a497249c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43117 } }
I20260811 12:08:14.142427 10997 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb0762c2b3794f39a1c948c4a497249c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:08:14.142544 10997 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb0762c2b3794f39a1c948c4a497249c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0762c2b3794f39a1c948c4a497249c, State: Initialized, Role: FOLLOWER
I20260811 12:08:14.143143 10997 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb0762c2b3794f39a1c948c4a497249c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb0762c2b3794f39a1c948c4a497249c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43117 } }
I20260811 12:08:14.143409 10997 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb0762c2b3794f39a1c948c4a497249c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:08:14.143563 10997 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb0762c2b3794f39a1c948c4a497249c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:08:14.143802 10997 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb0762c2b3794f39a1c948c4a497249c [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:08:14.146502 10997 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb0762c2b3794f39a1c948c4a497249c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb0762c2b3794f39a1c948c4a497249c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43117 } }
I20260811 12:08:14.147018 10997 leader_election.cc:302] T 00000000000000000000000000000000 P eb0762c2b3794f39a1c948c4a497249c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb0762c2b3794f39a1c948c4a497249c; no voters:
I20260811 12:08:14.147593 10997 leader_election.cc:288] T 00000000000000000000000000000000 P eb0762c2b3794f39a1c948c4a497249c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:08:14.147761 11019 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb0762c2b3794f39a1c948c4a497249c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:08:14.148531 11019 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb0762c2b3794f39a1c948c4a497249c [term 1 LEADER]: Becoming Leader. State: Replica: eb0762c2b3794f39a1c948c4a497249c, State: Running, Role: LEADER
I20260811 12:08:14.149137 11019 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb0762c2b3794f39a1c948c4a497249c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb0762c2b3794f39a1c948c4a497249c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43117 } }
I20260811 12:08:14.149848 10997 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb0762c2b3794f39a1c948c4a497249c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:08:14.152355 11022 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb0762c2b3794f39a1c948c4a497249c [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb0762c2b3794f39a1c948c4a497249c. Latest consensus state: current_term: 1 leader_uuid: "eb0762c2b3794f39a1c948c4a497249c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb0762c2b3794f39a1c948c4a497249c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43117 } } }
I20260811 12:08:14.152330 11021 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb0762c2b3794f39a1c948c4a497249c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb0762c2b3794f39a1c948c4a497249c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb0762c2b3794f39a1c948c4a497249c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43117 } } }
I20260811 12:08:14.152875 11022 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb0762c2b3794f39a1c948c4a497249c [sys.catalog]: This master's current role is: LEADER
I20260811 12:08:14.152930 11021 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb0762c2b3794f39a1c948c4a497249c [sys.catalog]: This master's current role is: LEADER
I20260811 12:08:14.155117 11028 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:08:14.160516 11028 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:08:14.170013 11028 catalog_manager.cc:1347] Generated new cluster ID: 9715bb8f051b494c9c7d5133a5e51f3c
I20260811 12:08:14.170135 11028 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:08:14.200129 11028 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:08:14.200593 11028 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:08:14.208778 11028 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb0762c2b3794f39a1c948c4a497249c: Generated new TSK 0
I20260811 12:08:14.209547 11028 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:08:14.215912 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 10896
W20260811 12:08:14.217346 10991 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.116:42655
W20260811 12:08:14.218081 10102 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 10896: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:08:14.218227 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 10896
2026-08-11T12:08:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2214 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449743084027-10102-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:08:14 dist-test-slave-mjjr krb5kdc[11051](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:08:14 dist-test-slave-mjjr krb5kdc[11051](info): set up 2 sockets
Aug 11 12:08:14 dist-test-slave-mjjr krb5kdc[11051](info): commencing operation
krb5kdc: starting...
W20260811 12:08:15.557403 10102 mini_kdc.cc:121] Time spent starting KDC: real 1.294s user 0.006s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Aug 11 12:08:15 dist-test-slave-mjjr krb5kdc[11051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:08:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:08:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449743084027-10102-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449743084027-10102-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 12:08:15.862068 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42331
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449743084027-10102-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42331
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:08:16.199208 11070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:08:16.199512 11070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:08:16.199687 11070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:08:16.206669 11070 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:08:16.206801 11070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:08:16.206899 11070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:08:16.207015 11070 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:08:16.207090 11070 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:08:16.216547 11070 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42331
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449743084027-10102-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42331
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:08:16.217724 11070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:08:16.218969 11070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:08:16.230782 11079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:16.230746 11080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:16.230971 11082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:08:16.231484 11070 server_base.cc:1034] running on GCE node
I20260811 12:08:16.232251 11070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:08:16.233532 11070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:08:16.234747 11070 hybrid_clock.cc:648] HybridClock initialized: now 1786450096234708 us; error 45 us; skew 500 ppm
I20260811 12:08:16.235082 11070 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:08:16 dist-test-slave-mjjr krb5kdc[11051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450096, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:08:16.238142 11070 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 12:08:16.239607 11070 webserver.cc:460] Webserver started at http://127.0.0.1:38351/ using document root <none> and password file <none>
I20260811 12:08:16.240291 11070 fs_manager.cc:359] Metadata directory not provided
I20260811 12:08:16.240444 11070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:08:16.240731 11070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:08:16.243620 11070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "311c05dfa68a4045b9ca0a04d4c9f5a1"
format_stamp: "Formatted at 2026-08-11 12:08:16 on dist-test-slave-mjjr"
server_key: "03b1a0de548b0959f5ed585cc47a695d"
server_key_iv: "db71ebe399903d0b78321c1e4a3f6bc9"
server_key_version: "encryptionkey@0"
I20260811 12:08:16.244453 11070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "311c05dfa68a4045b9ca0a04d4c9f5a1"
format_stamp: "Formatted at 2026-08-11 12:08:16 on dist-test-slave-mjjr"
server_key: "03b1a0de548b0959f5ed585cc47a695d"
server_key_iv: "db71ebe399903d0b78321c1e4a3f6bc9"
server_key_version: "encryptionkey@0"
I20260811 12:08:16.249311 11070 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 12:08:16.252689 11098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:08:16.253933 11070 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 12:08:16.254127 11070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "311c05dfa68a4045b9ca0a04d4c9f5a1"
format_stamp: "Formatted at 2026-08-11 12:08:16 on dist-test-slave-mjjr"
server_key: "03b1a0de548b0959f5ed585cc47a695d"
server_key_iv: "db71ebe399903d0b78321c1e4a3f6bc9"
server_key_version: "encryptionkey@0"
I20260811 12:08:16.254314 11070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:08:16.274930 11070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:08:16.278931 11070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:08:16.279224 11070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:08:16.298224 11070 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42331
I20260811 12:08:16.298280 11175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42331 every 8 connection(s)
I20260811 12:08:16.299588 11070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:08:16.300689 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 11070
I20260811 12:08:16.300874 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:08:16.301453 10102 external_mini_cluster.cc:1428] Setting key 299b8af47ea12373dfc77276ee504377
I20260811 12:08:16.304122 11177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 12:08:16 dist-test-slave-mjjr krb5kdc[11051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 12:08:16.317718 11177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 311c05dfa68a4045b9ca0a04d4c9f5a1: Bootstrap starting.
I20260811 12:08:16.321741 11177 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 311c05dfa68a4045b9ca0a04d4c9f5a1: Neither blocks nor log segments found. Creating new log.
I20260811 12:08:16.322784 11177 log.cc:824] T 00000000000000000000000000000000 P 311c05dfa68a4045b9ca0a04d4c9f5a1: Log is configured to *not* fsync() on all Append() calls
I20260811 12:08:16.325279 11180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:16.303288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.116:57981 (local address 127.0.0.1:42331)
0811 12:08:16.304087 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:16.304098 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:16.304142 (+ 44us) server_negotiation.cc:407] Connection header received
0811 12:08:16.305808 (+ 1666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:16.305832 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:16.306177 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:16.306507 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:16.308284 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:16.309621 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:16.311385 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:16.312055 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:16.312178 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:16.317180 (+ 5002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:16.317191 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:16.317210 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:16.317259 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:16.321141 (+ 3882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:16.321944 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:16.321950 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:16.321956 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:16.322126 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:16.322608 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:16.322611 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:16.322613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:16.323164 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:16.323537 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:16.324069 (+ 532us) server_negotiation.cc:299] Negotiation successful
0811 12:08:16.324348 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":376,"threads_started":1}
I20260811 12:08:16.325837 11177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 311c05dfa68a4045b9ca0a04d4c9f5a1: No bootstrap required, opened a new log
I20260811 12:08:16.328920 10102 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:08:16.333441 11177 raft_consensus.cc:348] T 00000000000000000000000000000000 P 311c05dfa68a4045b9ca0a04d4c9f5a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "311c05dfa68a4045b9ca0a04d4c9f5a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42331 } }
I20260811 12:08:16.333840 11177 raft_consensus.cc:374] T 00000000000000000000000000000000 P 311c05dfa68a4045b9ca0a04d4c9f5a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:08:16.333952 11177 raft_consensus.cc:729] T 00000000000000000000000000000000 P 311c05dfa68a4045b9ca0a04d4c9f5a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 311c05dfa68a4045b9ca0a04d4c9f5a1, State: Initialized, Role: FOLLOWER
I20260811 12:08:16.334551 11177 consensus_queue.cc:260] T 00000000000000000000000000000000 P 311c05dfa68a4045b9ca0a04d4c9f5a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "311c05dfa68a4045b9ca0a04d4c9f5a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42331 } }
I20260811 12:08:16.334872 11177 raft_consensus.cc:388] T 00000000000000000000000000000000 P 311c05dfa68a4045b9ca0a04d4c9f5a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:08:16.335040 11177 raft_consensus.cc:482] T 00000000000000000000000000000000 P 311c05dfa68a4045b9ca0a04d4c9f5a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:08:16.335242 11177 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 311c05dfa68a4045b9ca0a04d4c9f5a1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:08:16.338196 11177 raft_consensus.cc:504] T 00000000000000000000000000000000 P 311c05dfa68a4045b9ca0a04d4c9f5a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "311c05dfa68a4045b9ca0a04d4c9f5a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42331 } }
I20260811 12:08:16.338727 11177 leader_election.cc:302] T 00000000000000000000000000000000 P 311c05dfa68a4045b9ca0a04d4c9f5a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 311c05dfa68a4045b9ca0a04d4c9f5a1; no voters:
I20260811 12:08:16.339403 11177 leader_election.cc:288] T 00000000000000000000000000000000 P 311c05dfa68a4045b9ca0a04d4c9f5a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:08:16.339545 11192 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 311c05dfa68a4045b9ca0a04d4c9f5a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:08:16.341854 11177 sys_catalog.cc:564] T 00000000000000000000000000000000 P 311c05dfa68a4045b9ca0a04d4c9f5a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:08:16.342425 11192 raft_consensus.cc:686] T 00000000000000000000000000000000 P 311c05dfa68a4045b9ca0a04d4c9f5a1 [term 1 LEADER]: Becoming Leader. State: Replica: 311c05dfa68a4045b9ca0a04d4c9f5a1, State: Running, Role: LEADER
I20260811 12:08:16.343081 11192 consensus_queue.cc:237] T 00000000000000000000000000000000 P 311c05dfa68a4045b9ca0a04d4c9f5a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "311c05dfa68a4045b9ca0a04d4c9f5a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42331 } }
I20260811 12:08:16.349299 11195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 311c05dfa68a4045b9ca0a04d4c9f5a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 311c05dfa68a4045b9ca0a04d4c9f5a1. Latest consensus state: current_term: 1 leader_uuid: "311c05dfa68a4045b9ca0a04d4c9f5a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "311c05dfa68a4045b9ca0a04d4c9f5a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42331 } } }
I20260811 12:08:16.349713 11195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 311c05dfa68a4045b9ca0a04d4c9f5a1 [sys.catalog]: This master's current role is: LEADER
I20260811 12:08:16.350428 11194 sys_catalog.cc:455] T 00000000000000000000000000000000 P 311c05dfa68a4045b9ca0a04d4c9f5a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "311c05dfa68a4045b9ca0a04d4c9f5a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "311c05dfa68a4045b9ca0a04d4c9f5a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42331 } } }
I20260811 12:08:16.350704 11194 sys_catalog.cc:458] T 00000000000000000000000000000000 P 311c05dfa68a4045b9ca0a04d4c9f5a1 [sys.catalog]: This master's current role is: LEADER
I20260811 12:08:16.351560 11180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:16.333293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.116:36171 (local address 127.0.0.1:42331)
0811 12:08:16.333491 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:16.333498 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:16.333955 (+ 457us) server_negotiation.cc:407] Connection header received
0811 12:08:16.334170 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:16.334174 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:16.334263 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:16.334371 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:16.336446 (+ 2075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:16.337240 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:16.338921 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:16.339373 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:16.339505 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:16.342205 (+ 2700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:16.342213 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:16.342220 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:16.342267 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:16.346799 (+ 4532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:16.347866 (+ 1067us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:16.347870 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:16.347872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:16.348028 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:16.348697 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:16.348705 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:16.348710 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:16.349162 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:16.349500 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:16.350809 (+ 1309us) server_negotiation.cc:299] Negotiation successful
0811 12:08:16.351022 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260811 12:08:16.364262 11218 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 311c05dfa68a4045b9ca0a04d4c9f5a1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 12:08:16.364405 11218 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 12:08:16.364516 11219 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:08:16.366086 11219 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:08:16.373540 11219 catalog_manager.cc:1347] Generated new cluster ID: 45bde7f3fd0e42cba30c5f069a148f6c
I20260811 12:08:16.373642 11219 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:08:16.403051 11219 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:08:16.404565 11219 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:08:16.415230 11219 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 311c05dfa68a4045b9ca0a04d4c9f5a1: Generated new TSK 0
I20260811 12:08:16.416121 11219 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:08:16.428378 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 11070
I20260811 12:08:16.429493 11171 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:08:16.522663 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 11070
W20260811 12:08:16.549796 11047 connection.cc:535] client connection to 127.0.0.1:42331 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42331): Connection reset by peer (error 104)
2026-08-11T12:08:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2310 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449743084027-10102-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:08:16 dist-test-slave-mjjr krb5kdc[11235](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:08:16 dist-test-slave-mjjr krb5kdc[11235](info): set up 2 sockets
Aug 11 12:08:16 dist-test-slave-mjjr krb5kdc[11235](info): commencing operation
krb5kdc: starting...
W20260811 12:08:17.855883 10102 mini_kdc.cc:121] Time spent starting KDC: real 1.282s user 0.003s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Aug 11 12:08:18 dist-test-slave-mjjr krb5kdc[11235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450098, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:08:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:08:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449743084027-10102-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449743084027-10102-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 12:08:18.152943 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45355
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449743084027-10102-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45355
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:08:18.491246 11261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:08:18.491564 11261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:08:18.491819 11261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:08:18.499096 11261 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:08:18.499245 11261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:08:18.499358 11261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:08:18.499470 11261 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:08:18.499549 11261 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:08:18.509413 11261 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45355
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449743084027-10102-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:08:18.510703 11261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:08:18.512034 11261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:08:18.523804 11268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:18.523849 11274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:18.523838 11276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:08:18.525061 11261 server_base.cc:1034] running on GCE node
I20260811 12:08:18.525878 11261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:08:18.527195 11261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:08:18.528426 11261 hybrid_clock.cc:648] HybridClock initialized: now 1786450098528380 us; error 54 us; skew 500 ppm
I20260811 12:08:18.528761 11261 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:08:18 dist-test-slave-mjjr krb5kdc[11235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450098, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:08:18.531903 11261 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 12:08:18.533238 11261 webserver.cc:460] Webserver started at http://127.0.0.1:35423/ using document root <none> and password file <none>
I20260811 12:08:18.533897 11261 fs_manager.cc:359] Metadata directory not provided
I20260811 12:08:18.534029 11261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:08:18.534330 11261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:08:18.537317 11261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "64f622c257ae48a7b333a79476b356e9"
format_stamp: "Formatted at 2026-08-11 12:08:18 on dist-test-slave-mjjr"
server_key: "ec0e85c6dc4d80be1050cb3cf2470c67"
server_key_iv: "cf630bef57d76f37d28d62c9453b733f"
server_key_version: "encryptionkey@0"
I20260811 12:08:18.538069 11261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "64f622c257ae48a7b333a79476b356e9"
format_stamp: "Formatted at 2026-08-11 12:08:18 on dist-test-slave-mjjr"
server_key: "ec0e85c6dc4d80be1050cb3cf2470c67"
server_key_iv: "cf630bef57d76f37d28d62c9453b733f"
server_key_version: "encryptionkey@0"
I20260811 12:08:18.543064 11261 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:08:18.546424 11285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:08:18.547632 11261 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 12:08:18.547832 11261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "64f622c257ae48a7b333a79476b356e9"
format_stamp: "Formatted at 2026-08-11 12:08:18 on dist-test-slave-mjjr"
server_key: "ec0e85c6dc4d80be1050cb3cf2470c67"
server_key_iv: "cf630bef57d76f37d28d62c9453b733f"
server_key_version: "encryptionkey@0"
I20260811 12:08:18.548022 11261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:08:18.570048 11261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:08:18.573917 11261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:08:18.574210 11261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:08:18.593216 11261 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45355
I20260811 12:08:18.593248 11384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45355 every 8 connection(s)
I20260811 12:08:18.594612 11261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:08:18.598261 11385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:08:18.602571 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 11261
I20260811 12:08:18.602748 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:08:18.603329 10102 external_mini_cluster.cc:1428] Setting key c624afecf667aa943a7ae116d86d264d
I20260811 12:08:18.609830 11385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64f622c257ae48a7b333a79476b356e9: Bootstrap starting.
I20260811 12:08:18.613458 11385 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64f622c257ae48a7b333a79476b356e9: Neither blocks nor log segments found. Creating new log.
I20260811 12:08:18.614722 11385 log.cc:824] T 00000000000000000000000000000000 P 64f622c257ae48a7b333a79476b356e9: Log is configured to *not* fsync() on all Append() calls
I20260811 12:08:18.617872 11389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:18.605193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.116:57325 (local address 127.0.0.1:45355)
0811 12:08:18.605995 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:18.606006 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:18.606048 (+ 42us) server_negotiation.cc:407] Connection header received
0811 12:08:18.608462 (+ 2414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:18.608483 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:18.608868 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:18.609178 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:18.609811 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:18.611211 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:18.613124 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:18.613876 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:18.614005 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:18.615326 (+ 1321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:18.615340 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:18.615364 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:08:18.616181 (+ 817us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:18.616281 (+ 100us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:08:18.616307 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:18.616427 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:18.616785 (+ 358us) server_negotiation.cc:299] Negotiation successful
0811 12:08:18.616954 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":373,"threads_started":1}
I20260811 12:08:18.617903 11385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64f622c257ae48a7b333a79476b356e9: No bootstrap required, opened a new log
I20260811 12:08:18.621765 10102 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:08:18.627806 11385 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64f622c257ae48a7b333a79476b356e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64f622c257ae48a7b333a79476b356e9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45355 } }
I20260811 12:08:18.628299 11385 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64f622c257ae48a7b333a79476b356e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:08:18.628419 11385 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64f622c257ae48a7b333a79476b356e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f622c257ae48a7b333a79476b356e9, State: Initialized, Role: FOLLOWER
I20260811 12:08:18.629071 11385 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64f622c257ae48a7b333a79476b356e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64f622c257ae48a7b333a79476b356e9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45355 } }
I20260811 12:08:18.629361 11385 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64f622c257ae48a7b333a79476b356e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:08:18.629513 11385 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64f622c257ae48a7b333a79476b356e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:08:18.629699 11385 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64f622c257ae48a7b333a79476b356e9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:08:18.632341 11385 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64f622c257ae48a7b333a79476b356e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64f622c257ae48a7b333a79476b356e9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45355 } }
I20260811 12:08:18.632925 11385 leader_election.cc:302] T 00000000000000000000000000000000 P 64f622c257ae48a7b333a79476b356e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64f622c257ae48a7b333a79476b356e9; no voters:
I20260811 12:08:18.633599 11385 leader_election.cc:288] T 00000000000000000000000000000000 P 64f622c257ae48a7b333a79476b356e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:08:18.633751 11400 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64f622c257ae48a7b333a79476b356e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:08:18.633838 11389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:18.627142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.116:39195 (local address 127.0.0.1:45355)
0811 12:08:18.627316 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:18.627321 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:18.627777 (+ 456us) server_negotiation.cc:407] Connection header received
0811 12:08:18.627959 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:18.627964 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:18.628042 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:18.628156 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:18.628714 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:18.629856 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:18.631096 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:18.631818 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:18.631944 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:18.633016 (+ 1072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:18.633023 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:18.633028 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:08:18.633049 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:18.633094 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:08:18.633115 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:18.633233 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:18.633392 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 12:08:18.633465 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 12:08:18.634473 11400 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64f622c257ae48a7b333a79476b356e9 [term 1 LEADER]: Becoming Leader. State: Replica: 64f622c257ae48a7b333a79476b356e9, State: Running, Role: LEADER
I20260811 12:08:18.635056 11400 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64f622c257ae48a7b333a79476b356e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64f622c257ae48a7b333a79476b356e9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45355 } }
I20260811 12:08:18.636924 11400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64f622c257ae48a7b333a79476b356e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64f622c257ae48a7b333a79476b356e9. Latest consensus state: current_term: 1 leader_uuid: "64f622c257ae48a7b333a79476b356e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64f622c257ae48a7b333a79476b356e9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45355 } } }
I20260811 12:08:18.637326 11400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64f622c257ae48a7b333a79476b356e9 [sys.catalog]: This master's current role is: LEADER
I20260811 12:08:18.638173 11403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64f622c257ae48a7b333a79476b356e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "64f622c257ae48a7b333a79476b356e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64f622c257ae48a7b333a79476b356e9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45355 } } }
I20260811 12:08:18.638643 11403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64f622c257ae48a7b333a79476b356e9 [sys.catalog]: This master's current role is: LEADER
I20260811 12:08:18.641644 11385 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64f622c257ae48a7b333a79476b356e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:08:18.647032 11406 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:08:18.652521 11406 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:08:18.661146 11406 catalog_manager.cc:1347] Generated new cluster ID: 1dc8bd5caba94faaa30492ed3cc2662e
I20260811 12:08:18.661309 11406 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:08:18.682544 11406 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:08:18.683907 11406 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:08:18.690371 11406 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64f622c257ae48a7b333a79476b356e9: Generated new TSK 0
I20260811 12:08:18.691181 11406 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:08:18.705880 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 11261
W20260811 12:08:18.707154 11378 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.116:57325
W20260811 12:08:18.707973 10102 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 11261: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:08:18.708127 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 11261
W20260811 12:08:18.729115 11230 connection.cc:535] client connection to 127.0.0.1:45355 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45355): Connection reset by peer (error 104)
2026-08-11T12:08:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2174 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449743084027-10102-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:08:18 dist-test-slave-mjjr krb5kdc[11444](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:08:18 dist-test-slave-mjjr krb5kdc[11444](info): set up 2 sockets
Aug 11 12:08:18 dist-test-slave-mjjr krb5kdc[11444](info): commencing operation
krb5kdc: starting...
W20260811 12:08:20.053103 10102 mini_kdc.cc:121] Time spent starting KDC: real 1.304s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Aug 11 12:08:20 dist-test-slave-mjjr krb5kdc[11444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450100, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:08:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:08:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449743084027-10102-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449743084027-10102-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 12:08:20.351959 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42231
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449743084027-10102-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42231
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:08:20.693053 11459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:08:20.693367 11459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:08:20.693491 11459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:08:20.700500 11459 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:08:20.700639 11459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:08:20.700732 11459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:08:20.700832 11459 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:08:20.700913 11459 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:08:20.710390 11459 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42231
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449743084027-10102-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:08:20.711654 11459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:08:20.713241 11459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:08:20.724809 11471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:20.725244 11472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:20.725649 11475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:08:20.726452 11459 server_base.cc:1034] running on GCE node
I20260811 12:08:20.727183 11459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:08:20.728654 11459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:08:20.729863 11459 hybrid_clock.cc:648] HybridClock initialized: now 1786450100729786 us; error 52 us; skew 500 ppm
I20260811 12:08:20.730302 11459 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:08:20 dist-test-slave-mjjr krb5kdc[11444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450100, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:08:20.733494 11459 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 12:08:20.734977 11459 webserver.cc:460] Webserver started at http://127.0.0.1:45707/ using document root <none> and password file <none>
I20260811 12:08:20.735699 11459 fs_manager.cc:359] Metadata directory not provided
I20260811 12:08:20.735850 11459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:08:20.736140 11459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:08:20.739080 11459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "e90d73fa3b164026ad6481000e9dbf9d"
format_stamp: "Formatted at 2026-08-11 12:08:20 on dist-test-slave-mjjr"
server_key: "fbb1219188fbbb5d0f4d56ec5ce67c13"
server_key_iv: "16e3ba7e727014f259e3c8c063a06419"
server_key_version: "encryptionkey@0"
I20260811 12:08:20.739908 11459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "e90d73fa3b164026ad6481000e9dbf9d"
format_stamp: "Formatted at 2026-08-11 12:08:20 on dist-test-slave-mjjr"
server_key: "fbb1219188fbbb5d0f4d56ec5ce67c13"
server_key_iv: "16e3ba7e727014f259e3c8c063a06419"
server_key_version: "encryptionkey@0"
I20260811 12:08:20.745061 11459 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 12:08:20.748565 11485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:08:20.749838 11459 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 12:08:20.750041 11459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "e90d73fa3b164026ad6481000e9dbf9d"
format_stamp: "Formatted at 2026-08-11 12:08:20 on dist-test-slave-mjjr"
server_key: "fbb1219188fbbb5d0f4d56ec5ce67c13"
server_key_iv: "16e3ba7e727014f259e3c8c063a06419"
server_key_version: "encryptionkey@0"
I20260811 12:08:20.750257 11459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:08:20.755997 11459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:08:20.760035 11459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:08:20.760360 11459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:08:20.780068 11459 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42231
I20260811 12:08:20.780112 11590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42231 every 8 connection(s)
I20260811 12:08:20.781527 11459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:08:20.784976 11593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:08:20.789602 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 11459
I20260811 12:08:20.789834 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:08:20.790439 10102 external_mini_cluster.cc:1428] Setting key d19b0bbba2d1917725677cc676cc5639
I20260811 12:08:20.796962 11593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e90d73fa3b164026ad6481000e9dbf9d: Bootstrap starting.
I20260811 12:08:20.798503 11596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:20.792443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.116:42809 (local address 127.0.0.1:42231)
0811 12:08:20.793238 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:20.793250 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:20.793293 (+ 43us) server_negotiation.cc:407] Connection header received
0811 12:08:20.795164 (+ 1871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:20.795185 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:20.795506 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:20.795852 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:20.796974 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:20.796980 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:20.796991 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:08:20.797004 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:20.797088 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:08:20.797125 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:20.797286 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:20.797435 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 12:08:20.797613 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":386,"threads_started":1}
I20260811 12:08:20.800662 11593 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e90d73fa3b164026ad6481000e9dbf9d: Neither blocks nor log segments found. Creating new log.
I20260811 12:08:20.802019 11593 log.cc:824] T 00000000000000000000000000000000 P e90d73fa3b164026ad6481000e9dbf9d: Log is configured to *not* fsync() on all Append() calls
I20260811 12:08:20.802479 10102 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:08:20.804903 11593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e90d73fa3b164026ad6481000e9dbf9d: No bootstrap required, opened a new log
I20260811 12:08:20.809715 11596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:20.807114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.116:37607 (local address 127.0.0.1:42231)
0811 12:08:20.807330 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:20.807341 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:20.807716 (+ 375us) server_negotiation.cc:407] Connection header received
0811 12:08:20.807887 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:20.807891 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:20.807932 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:20.808008 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:20.809058 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:20.809061 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:20.809065 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:08:20.809079 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:20.809117 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:08:20.809130 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:20.809217 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:20.809371 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 12:08:20.809420 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 12:08:20.814090 11593 raft_consensus.cc:348] T 00000000000000000000000000000000 P e90d73fa3b164026ad6481000e9dbf9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e90d73fa3b164026ad6481000e9dbf9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42231 } }
I20260811 12:08:20.814482 11593 raft_consensus.cc:374] T 00000000000000000000000000000000 P e90d73fa3b164026ad6481000e9dbf9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:08:20.814605 11593 raft_consensus.cc:729] T 00000000000000000000000000000000 P e90d73fa3b164026ad6481000e9dbf9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90d73fa3b164026ad6481000e9dbf9d, State: Initialized, Role: FOLLOWER
I20260811 12:08:20.815162 11593 consensus_queue.cc:260] T 00000000000000000000000000000000 P e90d73fa3b164026ad6481000e9dbf9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e90d73fa3b164026ad6481000e9dbf9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42231 } }
I20260811 12:08:20.815433 11593 raft_consensus.cc:388] T 00000000000000000000000000000000 P e90d73fa3b164026ad6481000e9dbf9d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:08:20.815598 11593 raft_consensus.cc:482] T 00000000000000000000000000000000 P e90d73fa3b164026ad6481000e9dbf9d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:08:20.815848 11593 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e90d73fa3b164026ad6481000e9dbf9d [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:08:20.818356 11593 raft_consensus.cc:504] T 00000000000000000000000000000000 P e90d73fa3b164026ad6481000e9dbf9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e90d73fa3b164026ad6481000e9dbf9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42231 } }
I20260811 12:08:20.818903 11593 leader_election.cc:302] T 00000000000000000000000000000000 P e90d73fa3b164026ad6481000e9dbf9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e90d73fa3b164026ad6481000e9dbf9d; no voters:
I20260811 12:08:20.819490 11593 leader_election.cc:288] T 00000000000000000000000000000000 P e90d73fa3b164026ad6481000e9dbf9d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:08:20.819591 11610 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e90d73fa3b164026ad6481000e9dbf9d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:08:20.820461 11610 raft_consensus.cc:686] T 00000000000000000000000000000000 P e90d73fa3b164026ad6481000e9dbf9d [term 1 LEADER]: Becoming Leader. State: Replica: e90d73fa3b164026ad6481000e9dbf9d, State: Running, Role: LEADER
I20260811 12:08:20.821094 11610 consensus_queue.cc:237] T 00000000000000000000000000000000 P e90d73fa3b164026ad6481000e9dbf9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e90d73fa3b164026ad6481000e9dbf9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42231 } }
I20260811 12:08:20.821933 11593 sys_catalog.cc:564] T 00000000000000000000000000000000 P e90d73fa3b164026ad6481000e9dbf9d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:08:20.825773 11614 sys_catalog.cc:455] T 00000000000000000000000000000000 P e90d73fa3b164026ad6481000e9dbf9d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e90d73fa3b164026ad6481000e9dbf9d. Latest consensus state: current_term: 1 leader_uuid: "e90d73fa3b164026ad6481000e9dbf9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e90d73fa3b164026ad6481000e9dbf9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42231 } } }
I20260811 12:08:20.830982 11614 sys_catalog.cc:458] T 00000000000000000000000000000000 P e90d73fa3b164026ad6481000e9dbf9d [sys.catalog]: This master's current role is: LEADER
I20260811 12:08:20.825438 11613 sys_catalog.cc:455] T 00000000000000000000000000000000 P e90d73fa3b164026ad6481000e9dbf9d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e90d73fa3b164026ad6481000e9dbf9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e90d73fa3b164026ad6481000e9dbf9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42231 } } }
I20260811 12:08:20.831665 11613 sys_catalog.cc:458] T 00000000000000000000000000000000 P e90d73fa3b164026ad6481000e9dbf9d [sys.catalog]: This master's current role is: LEADER
I20260811 12:08:20.834064 11626 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:08:20.839128 11626 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:08:20.848232 11626 catalog_manager.cc:1347] Generated new cluster ID: ccd0c24d563742398628218090db5b94
I20260811 12:08:20.848340 11626 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:08:20.892225 11626 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:08:20.892639 11626 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:08:20.911118 11626 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e90d73fa3b164026ad6481000e9dbf9d: Generated new TSK 0
I20260811 12:08:20.912009 11626 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:08:20.920454 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 11459
W20260811 12:08:20.921891 11581 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.116:42809
W20260811 12:08:20.922753 10102 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 11459: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:08:20.922969 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 11459
2026-08-11T12:08:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2218 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449743084027-10102-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:08:21 dist-test-slave-mjjr krb5kdc[11653](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:08:21 dist-test-slave-mjjr krb5kdc[11653](info): set up 2 sockets
Aug 11 12:08:21 dist-test-slave-mjjr krb5kdc[11653](info): commencing operation
krb5kdc: starting...
W20260811 12:08:22.280942 10102 mini_kdc.cc:121] Time spent starting KDC: real 1.313s user 0.007s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Aug 11 12:08:22 dist-test-slave-mjjr krb5kdc[11653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:08:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:08:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.221.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
I20260811 12:08:22.580999 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:41667
--webserver_interface=127.9.221.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:40785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:41667
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:08:22.922986 11682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:08:22.923334 11682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:08:22.923494 11682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:08:22.931324 11682 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:08:22.931475 11682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:08:22.931582 11682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:08:22.931712 11682 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:08:22.931802 11682 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:08:22.942239 11682 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:40785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:41667
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:41667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.221.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:08:22.943624 11682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:08:22.945039 11682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:08:22.957449 11687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:22.957640 11690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:22.957840 11694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:08:22.958492 11682 server_base.cc:1034] running on GCE node
I20260811 12:08:22.959240 11682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:08:22.960620 11682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:08:22.961848 11682 hybrid_clock.cc:648] HybridClock initialized: now 1786450102961810 us; error 53 us; skew 500 ppm
I20260811 12:08:22.962201 11682 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:08:22 dist-test-slave-mjjr krb5kdc[11653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450102, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:08:22.965404 11682 init.cc:377] Logged in from keytab as kudu/127.9.221.190@KRBTEST.COM (short username kudu)
I20260811 12:08:22.966919 11682 webserver.cc:460] Webserver started at http://127.9.221.190:40479/ using document root <none> and password file <none>
I20260811 12:08:22.967723 11682 fs_manager.cc:359] Metadata directory not provided
I20260811 12:08:22.967875 11682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:08:22.968189 11682 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:08:22.971133 11682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "b88eecda0f5d41e2bdf4b112f85e5493"
format_stamp: "Formatted at 2026-08-11 12:08:22 on dist-test-slave-mjjr"
server_key: "913069156e23365651cf9d103877f3d1"
server_key_iv: "b43125ca880527ccb5887b315a0cea2a"
server_key_version: "encryptionkey@0"
I20260811 12:08:22.972005 11682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "b88eecda0f5d41e2bdf4b112f85e5493"
format_stamp: "Formatted at 2026-08-11 12:08:22 on dist-test-slave-mjjr"
server_key: "913069156e23365651cf9d103877f3d1"
server_key_iv: "b43125ca880527ccb5887b315a0cea2a"
server_key_version: "encryptionkey@0"
I20260811 12:08:22.976953 11682 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 12:08:22.980249 11705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:08:22.981585 11682 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:08:22.981767 11682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "b88eecda0f5d41e2bdf4b112f85e5493"
format_stamp: "Formatted at 2026-08-11 12:08:22 on dist-test-slave-mjjr"
server_key: "913069156e23365651cf9d103877f3d1"
server_key_iv: "b43125ca880527ccb5887b315a0cea2a"
server_key_version: "encryptionkey@0"
I20260811 12:08:22.981958 11682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:08:23.019907 11682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:08:23.024324 11682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:08:23.024631 11682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:08:23.044353 11682 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.190:41667
I20260811 12:08:23.044401 11797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.190:41667 every 8 connection(s)
I20260811 12:08:23.045732 11682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:08:23.049774 11800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:08:23.052663 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 11682
I20260811 12:08:23.052860 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:08:23.053284 10102 external_mini_cluster.cc:1428] Setting key bb1a433f44091c7c7be5b73a125dd9fb
I20260811 12:08:23.061738 11800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b88eecda0f5d41e2bdf4b112f85e5493: Bootstrap starting.
I20260811 12:08:23.064882 11800 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b88eecda0f5d41e2bdf4b112f85e5493: Neither blocks nor log segments found. Creating new log.
I20260811 12:08:23.065929 11800 log.cc:824] T 00000000000000000000000000000000 P b88eecda0f5d41e2bdf4b112f85e5493: Log is configured to *not* fsync() on all Append() calls
Aug 11 12:08:23 dist-test-slave-mjjr krb5kdc[11653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:08:23.069355 11800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b88eecda0f5d41e2bdf4b112f85e5493: No bootstrap required, opened a new log
I20260811 12:08:23.076267 11808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:23.055212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52621 (local address 127.9.221.190:41667)
0811 12:08:23.056056 (+ 844us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:23.056067 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:23.056106 (+ 39us) server_negotiation.cc:407] Connection header received
0811 12:08:23.057788 (+ 1682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:23.057808 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:23.058183 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:23.058558 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:23.060057 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:23.061340 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:23.062644 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:23.063287 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:23.063494 (+ 207us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:23.068336 (+ 4842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:23.068345 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:23.068358 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:23.068393 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:23.071733 (+ 3340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:23.072764 (+ 1031us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:23.072769 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:23.072779 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:23.072937 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:23.073458 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:23.073462 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:23.073464 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:23.073983 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:23.074220 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:23.074780 (+ 560us) server_negotiation.cc:299] Negotiation successful
0811 12:08:23.075031 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":395,"threads_started":1}
I20260811 12:08:23.079448 11800 raft_consensus.cc:348] T 00000000000000000000000000000000 P b88eecda0f5d41e2bdf4b112f85e5493 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b88eecda0f5d41e2bdf4b112f85e5493" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 41667 } }
I20260811 12:08:23.079948 11800 raft_consensus.cc:374] T 00000000000000000000000000000000 P b88eecda0f5d41e2bdf4b112f85e5493 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:08:23.080080 11800 raft_consensus.cc:729] T 00000000000000000000000000000000 P b88eecda0f5d41e2bdf4b112f85e5493 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b88eecda0f5d41e2bdf4b112f85e5493, State: Initialized, Role: FOLLOWER
I20260811 12:08:23.080401 10102 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:08:23.080829 11800 consensus_queue.cc:260] T 00000000000000000000000000000000 P b88eecda0f5d41e2bdf4b112f85e5493 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b88eecda0f5d41e2bdf4b112f85e5493" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 41667 } }
I20260811 12:08:23.081117 11800 raft_consensus.cc:388] T 00000000000000000000000000000000 P b88eecda0f5d41e2bdf4b112f85e5493 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:08:23.081259 11800 raft_consensus.cc:482] T 00000000000000000000000000000000 P b88eecda0f5d41e2bdf4b112f85e5493 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:08:23.081439 11800 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b88eecda0f5d41e2bdf4b112f85e5493 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:08:23.085234 11800 raft_consensus.cc:504] T 00000000000000000000000000000000 P b88eecda0f5d41e2bdf4b112f85e5493 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b88eecda0f5d41e2bdf4b112f85e5493" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 41667 } }
I20260811 12:08:23.085940 11800 leader_election.cc:302] T 00000000000000000000000000000000 P b88eecda0f5d41e2bdf4b112f85e5493 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b88eecda0f5d41e2bdf4b112f85e5493; no voters:
I20260811 12:08:23.086722 11800 leader_election.cc:288] T 00000000000000000000000000000000 P b88eecda0f5d41e2bdf4b112f85e5493 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:08:23.086834 11816 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b88eecda0f5d41e2bdf4b112f85e5493 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:08:23.089656 11800 sys_catalog.cc:564] T 00000000000000000000000000000000 P b88eecda0f5d41e2bdf4b112f85e5493 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:08:23.093773 11816 raft_consensus.cc:686] T 00000000000000000000000000000000 P b88eecda0f5d41e2bdf4b112f85e5493 [term 1 LEADER]: Becoming Leader. State: Replica: b88eecda0f5d41e2bdf4b112f85e5493, State: Running, Role: LEADER
I20260811 12:08:23.094623 11816 consensus_queue.cc:237] T 00000000000000000000000000000000 P b88eecda0f5d41e2bdf4b112f85e5493 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b88eecda0f5d41e2bdf4b112f85e5493" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 41667 } }
I20260811 12:08:23.098207 11819 sys_catalog.cc:455] T 00000000000000000000000000000000 P b88eecda0f5d41e2bdf4b112f85e5493 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b88eecda0f5d41e2bdf4b112f85e5493" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b88eecda0f5d41e2bdf4b112f85e5493" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 41667 } } }
I20260811 12:08:23.098803 11819 sys_catalog.cc:458] T 00000000000000000000000000000000 P b88eecda0f5d41e2bdf4b112f85e5493 [sys.catalog]: This master's current role is: LEADER
I20260811 12:08:23.098940 11820 sys_catalog.cc:455] T 00000000000000000000000000000000 P b88eecda0f5d41e2bdf4b112f85e5493 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b88eecda0f5d41e2bdf4b112f85e5493. Latest consensus state: current_term: 1 leader_uuid: "b88eecda0f5d41e2bdf4b112f85e5493" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b88eecda0f5d41e2bdf4b112f85e5493" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 41667 } } }
I20260811 12:08:23.099447 11820 sys_catalog.cc:458] T 00000000000000000000000000000000 P b88eecda0f5d41e2bdf4b112f85e5493 [sys.catalog]: This master's current role is: LEADER
I20260811 12:08:23.106719 11830 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:08:23.112195 11830 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:08:23.115530 11808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:23.086222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33877 (local address 127.9.221.190:41667)
0811 12:08:23.086400 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:23.086409 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:23.093136 (+ 6727us) server_negotiation.cc:407] Connection header received
0811 12:08:23.093406 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:23.093413 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:23.093493 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:23.093610 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:23.095321 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:23.096233 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:23.098075 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:23.098898 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:23.099086 (+ 188us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:23.105771 (+ 6685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:23.105784 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:23.105789 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:23.105828 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:23.110597 (+ 4769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:23.111455 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:23.111463 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:23.111483 (+ 20us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:23.111643 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:23.112144 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:23.112152 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:23.112157 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:23.112667 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:23.113022 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:23.114615 (+ 1593us) server_negotiation.cc:299] Negotiation successful
0811 12:08:23.114889 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 12:08:23.121563 11830 catalog_manager.cc:1347] Generated new cluster ID: b7d5b3ccb5c54be9bc2938a9b08a7c70
I20260811 12:08:23.121692 11830 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:08:23.149988 11830 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:08:23.151846 11830 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:08:23.159951 11830 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b88eecda0f5d41e2bdf4b112f85e5493: Generated new TSK 0
I20260811 12:08:23.160758 11830 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:08:23.169495 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 11682
I20260811 12:08:23.171069 11787 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:08:23.261405 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 11682
W20260811 12:08:23.284631 11649 connection.cc:535] client connection to 127.9.221.190:41667 recv error: Network error: failed to read from TLS socket (remote: 127.9.221.190:41667): Connection reset by peer (error 104)
2026-08-11T12:08:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2336 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449743084027-10102-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:08:23 dist-test-slave-mjjr krb5kdc[11850](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:08:23 dist-test-slave-mjjr krb5kdc[11850](info): set up 2 sockets
Aug 11 12:08:23 dist-test-slave-mjjr krb5kdc[11850](info): commencing operation
krb5kdc: starting...
W20260811 12:08:24.591220 10102 mini_kdc.cc:121] Time spent starting KDC: real 1.288s user 0.004s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Aug 11 12:08:24 dist-test-slave-mjjr krb5kdc[11850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450104, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:08:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:08:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.221.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
I20260811 12:08:24.897020 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:33885
--webserver_interface=127.9.221.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:34119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:33885
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:08:25.235937 11870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:08:25.236212 11870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:08:25.236397 11870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:08:25.243613 11870 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:08:25.243796 11870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:08:25.243909 11870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:08:25.244025 11870 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:08:25.244103 11870 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:08:25.254230 11870 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:34119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:33885
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:33885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.221.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:08:25.255478 11870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:08:25.256808 11870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:08:25.268626 11878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:25.268832 11877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:25.268841 11884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:08:25.270064 11870 server_base.cc:1034] running on GCE node
I20260811 12:08:25.270920 11870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:08:25.272363 11870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:08:25.273612 11870 hybrid_clock.cc:648] HybridClock initialized: now 1786450105273574 us; error 48 us; skew 500 ppm
I20260811 12:08:25.273973 11870 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:08:25 dist-test-slave-mjjr krb5kdc[11850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450105, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:08:25.277186 11870 init.cc:377] Logged in from keytab as kudu/127.9.221.190@KRBTEST.COM (short username kudu)
I20260811 12:08:25.278672 11870 webserver.cc:460] Webserver started at http://127.9.221.190:35571/ using document root <none> and password file <none>
I20260811 12:08:25.279381 11870 fs_manager.cc:359] Metadata directory not provided
I20260811 12:08:25.279533 11870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:08:25.279901 11870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:08:25.282936 11870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "b073a98ec4b440b298298ee38df3e052"
format_stamp: "Formatted at 2026-08-11 12:08:25 on dist-test-slave-mjjr"
server_key: "79d0013b31e24eb1ed3d432594e4c20f"
server_key_iv: "376f478aee3f951eda7f55e60f8c2f0a"
server_key_version: "encryptionkey@0"
I20260811 12:08:25.283807 11870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "b073a98ec4b440b298298ee38df3e052"
format_stamp: "Formatted at 2026-08-11 12:08:25 on dist-test-slave-mjjr"
server_key: "79d0013b31e24eb1ed3d432594e4c20f"
server_key_iv: "376f478aee3f951eda7f55e60f8c2f0a"
server_key_version: "encryptionkey@0"
I20260811 12:08:25.289182 11870 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 12:08:25.292520 11898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:08:25.293723 11870 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:08:25.293910 11870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "b073a98ec4b440b298298ee38df3e052"
format_stamp: "Formatted at 2026-08-11 12:08:25 on dist-test-slave-mjjr"
server_key: "79d0013b31e24eb1ed3d432594e4c20f"
server_key_iv: "376f478aee3f951eda7f55e60f8c2f0a"
server_key_version: "encryptionkey@0"
I20260811 12:08:25.294099 11870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:08:25.318295 11870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:08:25.322391 11870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:08:25.322690 11870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:08:25.341753 11870 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.190:33885
I20260811 12:08:25.341773 11975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.190:33885 every 8 connection(s)
I20260811 12:08:25.343225 11870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:08:25.347111 11977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:08:25.349152 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 11870
I20260811 12:08:25.349401 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:08:25.349949 10102 external_mini_cluster.cc:1428] Setting key 53fa2b111bc8649bc717690fbecee825
I20260811 12:08:25.360543 11977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b073a98ec4b440b298298ee38df3e052: Bootstrap starting.
I20260811 12:08:25.364593 11977 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b073a98ec4b440b298298ee38df3e052: Neither blocks nor log segments found. Creating new log.
I20260811 12:08:25.365792 11985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:25.352152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43547 (local address 127.9.221.190:33885)
0811 12:08:25.353027 (+ 875us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:25.353042 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:25.353094 (+ 52us) server_negotiation.cc:407] Connection header received
0811 12:08:25.355488 (+ 2394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:25.355518 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:25.356032 (+ 514us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:25.356487 (+ 455us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:25.357081 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:25.359173 (+ 2092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:25.360573 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:25.361577 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:25.361767 (+ 190us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:25.362924 (+ 1157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:25.362935 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:25.362951 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:08:25.363683 (+ 732us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:25.363780 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:08:25.363812 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:25.363986 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:25.364243 (+ 257us) server_negotiation.cc:299] Negotiation successful
0811 12:08:25.364413 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":358,"threads_started":1}
I20260811 12:08:25.365944 11977 log.cc:824] T 00000000000000000000000000000000 P b073a98ec4b440b298298ee38df3e052: Log is configured to *not* fsync() on all Append() calls
I20260811 12:08:25.369269 11977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b073a98ec4b440b298298ee38df3e052: No bootstrap required, opened a new log
I20260811 12:08:25.369944 10102 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:08:25.377942 11977 raft_consensus.cc:348] T 00000000000000000000000000000000 P b073a98ec4b440b298298ee38df3e052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b073a98ec4b440b298298ee38df3e052" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 33885 } }
I20260811 12:08:25.378307 11977 raft_consensus.cc:374] T 00000000000000000000000000000000 P b073a98ec4b440b298298ee38df3e052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:08:25.378407 11977 raft_consensus.cc:729] T 00000000000000000000000000000000 P b073a98ec4b440b298298ee38df3e052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b073a98ec4b440b298298ee38df3e052, State: Initialized, Role: FOLLOWER
I20260811 12:08:25.379055 11977 consensus_queue.cc:260] T 00000000000000000000000000000000 P b073a98ec4b440b298298ee38df3e052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b073a98ec4b440b298298ee38df3e052" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 33885 } }
I20260811 12:08:25.379346 11977 raft_consensus.cc:388] T 00000000000000000000000000000000 P b073a98ec4b440b298298ee38df3e052 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:08:25.379501 11977 raft_consensus.cc:482] T 00000000000000000000000000000000 P b073a98ec4b440b298298ee38df3e052 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:08:25.379688 11977 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b073a98ec4b440b298298ee38df3e052 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:08:25.381199 11985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:25.374419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46963 (local address 127.9.221.190:33885)
0811 12:08:25.374598 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:25.374604 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:25.374987 (+ 383us) server_negotiation.cc:407] Connection header received
0811 12:08:25.375116 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:25.375121 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:25.375196 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:25.375308 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:25.375809 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:25.377134 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:25.378509 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:25.378977 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:25.379131 (+ 154us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:25.380445 (+ 1314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:25.380449 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:25.380452 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:08:25.380468 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:25.380527 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:08:25.380555 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:25.380694 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:25.380875 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 12:08:25.380929 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 12:08:25.382800 11977 raft_consensus.cc:504] T 00000000000000000000000000000000 P b073a98ec4b440b298298ee38df3e052 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b073a98ec4b440b298298ee38df3e052" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 33885 } }
I20260811 12:08:25.383308 11977 leader_election.cc:302] T 00000000000000000000000000000000 P b073a98ec4b440b298298ee38df3e052 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b073a98ec4b440b298298ee38df3e052; no voters:
I20260811 12:08:25.383924 11977 leader_election.cc:288] T 00000000000000000000000000000000 P b073a98ec4b440b298298ee38df3e052 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:08:25.384043 12001 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b073a98ec4b440b298298ee38df3e052 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:08:25.384757 12001 raft_consensus.cc:686] T 00000000000000000000000000000000 P b073a98ec4b440b298298ee38df3e052 [term 1 LEADER]: Becoming Leader. State: Replica: b073a98ec4b440b298298ee38df3e052, State: Running, Role: LEADER
I20260811 12:08:25.385340 12001 consensus_queue.cc:237] T 00000000000000000000000000000000 P b073a98ec4b440b298298ee38df3e052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b073a98ec4b440b298298ee38df3e052" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 33885 } }
I20260811 12:08:25.385982 11977 sys_catalog.cc:564] T 00000000000000000000000000000000 P b073a98ec4b440b298298ee38df3e052 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:08:25.388306 12002 sys_catalog.cc:455] T 00000000000000000000000000000000 P b073a98ec4b440b298298ee38df3e052 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b073a98ec4b440b298298ee38df3e052" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b073a98ec4b440b298298ee38df3e052" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 33885 } } }
I20260811 12:08:25.388671 12002 sys_catalog.cc:458] T 00000000000000000000000000000000 P b073a98ec4b440b298298ee38df3e052 [sys.catalog]: This master's current role is: LEADER
I20260811 12:08:25.389592 12015 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:08:25.389411 12005 sys_catalog.cc:455] T 00000000000000000000000000000000 P b073a98ec4b440b298298ee38df3e052 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b073a98ec4b440b298298ee38df3e052. Latest consensus state: current_term: 1 leader_uuid: "b073a98ec4b440b298298ee38df3e052" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b073a98ec4b440b298298ee38df3e052" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 33885 } } }
I20260811 12:08:25.389813 12005 sys_catalog.cc:458] T 00000000000000000000000000000000 P b073a98ec4b440b298298ee38df3e052 [sys.catalog]: This master's current role is: LEADER
I20260811 12:08:25.395241 12015 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:08:25.404469 12015 catalog_manager.cc:1347] Generated new cluster ID: 848bd9a6cdf941078e707199a7a1d3a8
I20260811 12:08:25.404564 12015 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:08:25.423267 12015 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:08:25.424518 12015 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:08:25.435303 12015 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b073a98ec4b440b298298ee38df3e052: Generated new TSK 0
I20260811 12:08:25.436165 12015 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:08:25.445350 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 11870
W20260811 12:08:25.446499 11968 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:43547
W20260811 12:08:25.447237 10102 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 11870: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:08:25.447397 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 11870
W20260811 12:08:25.477634 11848 connection.cc:535] client connection to 127.9.221.190:33885 recv error: Network error: failed to read from TLS socket (remote: 127.9.221.190:33885): Connection reset by peer (error 104)
2026-08-11T12:08:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2192 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449743084027-10102-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:08:25 dist-test-slave-mjjr krb5kdc[12036](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:08:25 dist-test-slave-mjjr krb5kdc[12036](info): set up 2 sockets
Aug 11 12:08:25 dist-test-slave-mjjr krb5kdc[12036](info): commencing operation
krb5kdc: starting...
W20260811 12:08:26.799161 10102 mini_kdc.cc:121] Time spent starting KDC: real 1.302s user 0.002s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Aug 11 12:08:26 dist-test-slave-mjjr krb5kdc[12036](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:08:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:08:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.221.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
I20260811 12:08:27.104215 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:36313
--webserver_interface=127.9.221.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:45085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:36313
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:08:27.444061 12052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:08:27.444404 12052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:08:27.444572 12052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:08:27.451953 12052 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:08:27.452111 12052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:08:27.452214 12052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:08:27.452322 12052 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:08:27.452404 12052 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:08:27.462114 12052 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:45085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:36313
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:36313
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.221.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:08:27.463454 12052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:08:27.464844 12052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:08:27.477948 12067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:27.477916 12064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:27.477869 12063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:08:27.478927 12052 server_base.cc:1034] running on GCE node
I20260811 12:08:27.479775 12052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:08:27.481192 12052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:08:27.482420 12052 hybrid_clock.cc:648] HybridClock initialized: now 1786450107482384 us; error 48 us; skew 500 ppm
I20260811 12:08:27.482853 12052 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:08:27 dist-test-slave-mjjr krb5kdc[12036](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450107, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:08:27.486115 12052 init.cc:377] Logged in from keytab as kudu/127.9.221.190@KRBTEST.COM (short username kudu)
I20260811 12:08:27.487573 12052 webserver.cc:460] Webserver started at http://127.9.221.190:33009/ using document root <none> and password file <none>
I20260811 12:08:27.488303 12052 fs_manager.cc:359] Metadata directory not provided
I20260811 12:08:27.488466 12052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:08:27.488790 12052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:08:27.491868 12052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "2a41eb5776a742c0a0868dbaf34a1e3b"
format_stamp: "Formatted at 2026-08-11 12:08:27 on dist-test-slave-mjjr"
server_key: "bef465038831566dac86b50426c29fa1"
server_key_iv: "fb3e67e52e03971a8aeaf293348d379c"
server_key_version: "encryptionkey@0"
I20260811 12:08:27.492717 12052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "2a41eb5776a742c0a0868dbaf34a1e3b"
format_stamp: "Formatted at 2026-08-11 12:08:27 on dist-test-slave-mjjr"
server_key: "bef465038831566dac86b50426c29fa1"
server_key_iv: "fb3e67e52e03971a8aeaf293348d379c"
server_key_version: "encryptionkey@0"
I20260811 12:08:27.497972 12052 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:08:27.501442 12079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:08:27.502702 12052 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 12:08:27.502899 12052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "2a41eb5776a742c0a0868dbaf34a1e3b"
format_stamp: "Formatted at 2026-08-11 12:08:27 on dist-test-slave-mjjr"
server_key: "bef465038831566dac86b50426c29fa1"
server_key_iv: "fb3e67e52e03971a8aeaf293348d379c"
server_key_version: "encryptionkey@0"
I20260811 12:08:27.503108 12052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:08:27.508790 12052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:08:27.512991 12052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:08:27.513312 12052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:08:27.531756 12052 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.190:36313
I20260811 12:08:27.531759 12172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.190:36313 every 8 connection(s)
I20260811 12:08:27.533121 12052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:08:27.534087 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 12052
I20260811 12:08:27.534281 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:08:27.534719 10102 external_mini_cluster.cc:1428] Setting key 94de4f29a21b7c4786ac9f2e0ce8b58b
I20260811 12:08:27.537473 12173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:08:27.543380 12178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:27.536483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42223 (local address 127.9.221.190:36313)
0811 12:08:27.537308 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:27.537333 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:27.537381 (+ 48us) server_negotiation.cc:407] Connection header received
0811 12:08:27.539596 (+ 2215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:27.539620 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:27.540024 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:27.540519 (+ 495us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:27.541479 (+ 960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:27.541488 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:27.541504 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:08:27.541522 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:27.541606 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:08:27.541638 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:27.541873 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:27.542007 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 12:08:27.542210 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":313,"threads_started":1}
I20260811 12:08:27.546890 10102 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:08:27.550238 12173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a41eb5776a742c0a0868dbaf34a1e3b: Bootstrap starting.
I20260811 12:08:27.553929 12178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:27.551255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46501 (local address 127.9.221.190:36313)
0811 12:08:27.551460 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:27.551469 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:27.552005 (+ 536us) server_negotiation.cc:407] Connection header received
0811 12:08:27.552169 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:27.552173 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:27.552217 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:27.552294 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:27.553285 (+ 991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:27.553288 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:27.553291 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:08:27.553304 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:27.553338 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:08:27.553351 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:27.553435 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:27.553570 (+ 135us) server_negotiation.cc:299] Negotiation successful
0811 12:08:27.553611 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 12:08:27.554142 12173 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a41eb5776a742c0a0868dbaf34a1e3b: Neither blocks nor log segments found. Creating new log.
I20260811 12:08:27.555516 12173 log.cc:824] T 00000000000000000000000000000000 P 2a41eb5776a742c0a0868dbaf34a1e3b: Log is configured to *not* fsync() on all Append() calls
I20260811 12:08:27.558128 12173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a41eb5776a742c0a0868dbaf34a1e3b: No bootstrap required, opened a new log
I20260811 12:08:27.565793 12173 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a41eb5776a742c0a0868dbaf34a1e3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a41eb5776a742c0a0868dbaf34a1e3b" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 36313 } }
I20260811 12:08:27.566179 12173 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a41eb5776a742c0a0868dbaf34a1e3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:08:27.566284 12173 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a41eb5776a742c0a0868dbaf34a1e3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a41eb5776a742c0a0868dbaf34a1e3b, State: Initialized, Role: FOLLOWER
I20260811 12:08:27.566895 12173 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a41eb5776a742c0a0868dbaf34a1e3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a41eb5776a742c0a0868dbaf34a1e3b" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 36313 } }
I20260811 12:08:27.567130 12173 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a41eb5776a742c0a0868dbaf34a1e3b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:08:27.567282 12173 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a41eb5776a742c0a0868dbaf34a1e3b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:08:27.567471 12173 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a41eb5776a742c0a0868dbaf34a1e3b [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:08:27.569633 12173 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a41eb5776a742c0a0868dbaf34a1e3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a41eb5776a742c0a0868dbaf34a1e3b" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 36313 } }
I20260811 12:08:27.570170 12173 leader_election.cc:302] T 00000000000000000000000000000000 P 2a41eb5776a742c0a0868dbaf34a1e3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a41eb5776a742c0a0868dbaf34a1e3b; no voters:
I20260811 12:08:27.570753 12173 leader_election.cc:288] T 00000000000000000000000000000000 P 2a41eb5776a742c0a0868dbaf34a1e3b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:08:27.570842 12186 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a41eb5776a742c0a0868dbaf34a1e3b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:08:27.571702 12186 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a41eb5776a742c0a0868dbaf34a1e3b [term 1 LEADER]: Becoming Leader. State: Replica: 2a41eb5776a742c0a0868dbaf34a1e3b, State: Running, Role: LEADER
I20260811 12:08:27.572367 12186 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a41eb5776a742c0a0868dbaf34a1e3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a41eb5776a742c0a0868dbaf34a1e3b" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 36313 } }
I20260811 12:08:27.573096 12173 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a41eb5776a742c0a0868dbaf34a1e3b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:08:27.575549 12194 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a41eb5776a742c0a0868dbaf34a1e3b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a41eb5776a742c0a0868dbaf34a1e3b. Latest consensus state: current_term: 1 leader_uuid: "2a41eb5776a742c0a0868dbaf34a1e3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a41eb5776a742c0a0868dbaf34a1e3b" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 36313 } } }
I20260811 12:08:27.575568 12190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a41eb5776a742c0a0868dbaf34a1e3b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2a41eb5776a742c0a0868dbaf34a1e3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a41eb5776a742c0a0868dbaf34a1e3b" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 36313 } } }
I20260811 12:08:27.576342 12194 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a41eb5776a742c0a0868dbaf34a1e3b [sys.catalog]: This master's current role is: LEADER
I20260811 12:08:27.576483 12190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a41eb5776a742c0a0868dbaf34a1e3b [sys.catalog]: This master's current role is: LEADER
I20260811 12:08:27.578850 12200 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:08:27.583775 12200 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:08:27.593261 12200 catalog_manager.cc:1347] Generated new cluster ID: 25e2253ff30b4a32825f7a09f66009b2
I20260811 12:08:27.593403 12200 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:08:27.605060 12200 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:08:27.605500 12200 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:08:27.613425 12200 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a41eb5776a742c0a0868dbaf34a1e3b: Generated new TSK 0
I20260811 12:08:27.614246 12200 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:08:27.623744 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 12052
W20260811 12:08:27.625434 12165 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:42223
W20260811 12:08:27.626165 10102 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 12052: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:08:27.626331 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 12052
2026-08-11T12:08:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2176 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449743084027-10102-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:08:27 dist-test-slave-mjjr krb5kdc[12220](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:08:27 dist-test-slave-mjjr krb5kdc[12220](info): set up 2 sockets
Aug 11 12:08:27 dist-test-slave-mjjr krb5kdc[12220](info): commencing operation
krb5kdc: starting...
W20260811 12:08:28.980582 10102 mini_kdc.cc:121] Time spent starting KDC: real 1.307s user 0.001s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Aug 11 12:08:29 dist-test-slave-mjjr krb5kdc[12220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:08:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:08:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.221.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
I20260811 12:08:29.319396 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:40247
--webserver_interface=127.9.221.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:36211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:40247
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:08:29.658708 12253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:08:29.659065 12253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:08:29.659252 12253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:08:29.667105 12253 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:08:29.667264 12253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:08:29.667368 12253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:08:29.667479 12253 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:08:29.667565 12253 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:08:29.677377 12253 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:36211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:40247
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:40247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.221.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:08:29.678722 12253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:08:29.680117 12253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:08:29.692978 12260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:29.693015 12262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:29.692957 12267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:08:29.694025 12253 server_base.cc:1034] running on GCE node
I20260811 12:08:29.694754 12253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:08:29.696332 12253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:08:29.697561 12253 hybrid_clock.cc:648] HybridClock initialized: now 1786450109697531 us; error 56 us; skew 500 ppm
I20260811 12:08:29.697907 12253 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:08:29 dist-test-slave-mjjr krb5kdc[12220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450109, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:08:29.701464 12253 init.cc:377] Logged in from keytab as kudu/127.9.221.190@KRBTEST.COM (short username kudu)
I20260811 12:08:29.703159 12253 webserver.cc:460] Webserver started at http://127.9.221.190:35983/ using document root <none> and password file <none>
I20260811 12:08:29.703997 12253 fs_manager.cc:359] Metadata directory not provided
I20260811 12:08:29.704167 12253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:08:29.704501 12253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:08:29.707800 12253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "89642e2c50634400ac8121ab9ae5278c"
format_stamp: "Formatted at 2026-08-11 12:08:29 on dist-test-slave-mjjr"
server_key: "eba5c3e9563f449c5f0e88fd4e13d547"
server_key_iv: "536f1281631b08f20e1c7b537cc6db2f"
server_key_version: "encryptionkey@0"
I20260811 12:08:29.708635 12253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "89642e2c50634400ac8121ab9ae5278c"
format_stamp: "Formatted at 2026-08-11 12:08:29 on dist-test-slave-mjjr"
server_key: "eba5c3e9563f449c5f0e88fd4e13d547"
server_key_iv: "536f1281631b08f20e1c7b537cc6db2f"
server_key_version: "encryptionkey@0"
I20260811 12:08:29.714140 12253 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 12:08:29.717808 12280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:08:29.719085 12253 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:08:29.719285 12253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "89642e2c50634400ac8121ab9ae5278c"
format_stamp: "Formatted at 2026-08-11 12:08:29 on dist-test-slave-mjjr"
server_key: "eba5c3e9563f449c5f0e88fd4e13d547"
server_key_iv: "536f1281631b08f20e1c7b537cc6db2f"
server_key_version: "encryptionkey@0"
I20260811 12:08:29.719496 12253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:08:29.743889 12253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:08:29.747942 12253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:08:29.748255 12253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:08:29.767163 12253 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.190:40247
I20260811 12:08:29.767186 12366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.190:40247 every 8 connection(s)
I20260811 12:08:29.768595 12253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:08:29.770216 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 12253
I20260811 12:08:29.770398 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:08:29.770951 10102 external_mini_cluster.cc:1428] Setting key c18fe9c37c156eb67524a2d76439ff6d
I20260811 12:08:29.772872 12367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 12:08:29 dist-test-slave-mjjr krb5kdc[12220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:08:29.786458 12367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89642e2c50634400ac8121ab9ae5278c: Bootstrap starting.
I20260811 12:08:29.789954 12367 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89642e2c50634400ac8121ab9ae5278c: Neither blocks nor log segments found. Creating new log.
I20260811 12:08:29.791244 12367 log.cc:824] T 00000000000000000000000000000000 P 89642e2c50634400ac8121ab9ae5278c: Log is configured to *not* fsync() on all Append() calls
I20260811 12:08:29.794535 12367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89642e2c50634400ac8121ab9ae5278c: No bootstrap required, opened a new log
I20260811 12:08:29.794919 12371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:29.772684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33133 (local address 127.9.221.190:40247)
0811 12:08:29.773465 (+ 781us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:29.773479 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:29.773528 (+ 49us) server_negotiation.cc:407] Connection header received
0811 12:08:29.775764 (+ 2236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:29.775790 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:29.776315 (+ 525us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:29.776738 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:29.778072 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:29.780184 (+ 2112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:29.781440 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:29.782528 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:29.782726 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:29.787249 (+ 4523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:29.787257 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:29.787269 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:29.787303 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:29.790439 (+ 3136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:29.791315 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:29.791320 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:29.791325 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:29.791526 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:29.792035 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:29.792042 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:29.792049 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:29.792777 (+ 728us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:29.792992 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:29.793505 (+ 513us) server_negotiation.cc:299] Negotiation successful
0811 12:08:29.793845 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":352,"threads_started":1}
I20260811 12:08:29.798488 10102 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:08:29.802898 12367 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89642e2c50634400ac8121ab9ae5278c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89642e2c50634400ac8121ab9ae5278c" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40247 } }
I20260811 12:08:29.803354 12367 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89642e2c50634400ac8121ab9ae5278c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:08:29.803512 12367 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89642e2c50634400ac8121ab9ae5278c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89642e2c50634400ac8121ab9ae5278c, State: Initialized, Role: FOLLOWER
I20260811 12:08:29.804150 12367 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89642e2c50634400ac8121ab9ae5278c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89642e2c50634400ac8121ab9ae5278c" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40247 } }
I20260811 12:08:29.804399 12367 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89642e2c50634400ac8121ab9ae5278c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:08:29.804548 12367 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89642e2c50634400ac8121ab9ae5278c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:08:29.804723 12367 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89642e2c50634400ac8121ab9ae5278c [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:08:29.806970 12367 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89642e2c50634400ac8121ab9ae5278c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89642e2c50634400ac8121ab9ae5278c" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40247 } }
I20260811 12:08:29.807502 12367 leader_election.cc:302] T 00000000000000000000000000000000 P 89642e2c50634400ac8121ab9ae5278c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89642e2c50634400ac8121ab9ae5278c; no voters:
I20260811 12:08:29.808189 12367 leader_election.cc:288] T 00000000000000000000000000000000 P 89642e2c50634400ac8121ab9ae5278c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:08:29.808277 12392 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89642e2c50634400ac8121ab9ae5278c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:08:29.810621 12392 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89642e2c50634400ac8121ab9ae5278c [term 1 LEADER]: Becoming Leader. State: Replica: 89642e2c50634400ac8121ab9ae5278c, State: Running, Role: LEADER
I20260811 12:08:29.810707 12367 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89642e2c50634400ac8121ab9ae5278c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:08:29.811200 12392 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89642e2c50634400ac8121ab9ae5278c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89642e2c50634400ac8121ab9ae5278c" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40247 } }
I20260811 12:08:29.815479 12394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89642e2c50634400ac8121ab9ae5278c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "89642e2c50634400ac8121ab9ae5278c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89642e2c50634400ac8121ab9ae5278c" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40247 } } }
I20260811 12:08:29.816453 12394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89642e2c50634400ac8121ab9ae5278c [sys.catalog]: This master's current role is: LEADER
I20260811 12:08:29.815410 12393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89642e2c50634400ac8121ab9ae5278c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89642e2c50634400ac8121ab9ae5278c. Latest consensus state: current_term: 1 leader_uuid: "89642e2c50634400ac8121ab9ae5278c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89642e2c50634400ac8121ab9ae5278c" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40247 } } }
I20260811 12:08:29.817114 12393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89642e2c50634400ac8121ab9ae5278c [sys.catalog]: This master's current role is: LEADER
I20260811 12:08:29.820664 12371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:29.802683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46239 (local address 127.9.221.190:40247)
0811 12:08:29.802852 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:29.802858 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:29.803265 (+ 407us) server_negotiation.cc:407] Connection header received
0811 12:08:29.803450 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:29.803456 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:29.803527 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:29.803625 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:29.805156 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:29.805964 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:29.807172 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:29.807627 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:29.807798 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:29.810427 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:29.810438 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:29.810444 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:29.810488 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:29.815194 (+ 4706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:29.816292 (+ 1098us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:29.816301 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:29.816308 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:29.817563 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:29.818036 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:29.818044 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:29.818050 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:29.818551 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:29.818909 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:29.819848 (+ 939us) server_negotiation.cc:299] Negotiation successful
0811 12:08:29.820143 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 12:08:29.823828 12406 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:08:29.828713 12406 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:08:29.837412 12406 catalog_manager.cc:1347] Generated new cluster ID: 177f9ff72cd047b48045a39d6bce4a51
I20260811 12:08:29.837527 12406 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:08:29.848999 12406 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:08:29.850232 12406 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:08:29.859757 12406 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89642e2c50634400ac8121ab9ae5278c: Generated new TSK 0
I20260811 12:08:29.860653 12406 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:08:29.878666 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 12253
I20260811 12:08:29.879794 12360 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:08:29.971714 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 12253
2026-08-11T12:08:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2346 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449743084027-10102-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:08:30 dist-test-slave-mjjr krb5kdc[12428](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:08:30 dist-test-slave-mjjr krb5kdc[12428](info): set up 2 sockets
Aug 11 12:08:30 dist-test-slave-mjjr krb5kdc[12428](info): commencing operation
krb5kdc: starting...
W20260811 12:08:31.317860 10102 mini_kdc.cc:121] Time spent starting KDC: real 1.298s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Aug 11 12:08:31 dist-test-slave-mjjr krb5kdc[12428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450111, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:08:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:08:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.221.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
I20260811 12:08:31.625073 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:45175
--webserver_interface=127.9.221.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:33911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:45175
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:08:31.978855 12452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:08:31.979189 12452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:08:31.979341 12452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:08:31.986658 12452 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:08:31.986817 12452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:08:31.986913 12452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:08:31.987017 12452 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:08:31.987089 12452 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:08:31.996929 12452 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:33911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:45175
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:45175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.221.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:08:31.998211 12452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:08:31.999504 12452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:08:32.012645 12462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:32.012601 12466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:32.012610 12463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:08:32.013980 12452 server_base.cc:1034] running on GCE node
I20260811 12:08:32.014937 12452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:08:32.016964 12452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:08:32.018215 12452 hybrid_clock.cc:648] HybridClock initialized: now 1786450112018177 us; error 56 us; skew 500 ppm
I20260811 12:08:32.018635 12452 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:08:32 dist-test-slave-mjjr krb5kdc[12428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450112, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:08:32.021950 12452 init.cc:377] Logged in from keytab as kudu/127.9.221.190@KRBTEST.COM (short username kudu)
I20260811 12:08:32.023406 12452 webserver.cc:460] Webserver started at http://127.9.221.190:45095/ using document root <none> and password file <none>
I20260811 12:08:32.024163 12452 fs_manager.cc:359] Metadata directory not provided
I20260811 12:08:32.024323 12452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:08:32.024624 12452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:08:32.027577 12452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "fabdffa403f743c594919bf8bb0ef113"
format_stamp: "Formatted at 2026-08-11 12:08:32 on dist-test-slave-mjjr"
server_key: "66f546ca9374e58fc72107095314c011"
server_key_iv: "0e10a97b5246227d72359ee8ec92a0b0"
server_key_version: "encryptionkey@0"
I20260811 12:08:32.028456 12452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "fabdffa403f743c594919bf8bb0ef113"
format_stamp: "Formatted at 2026-08-11 12:08:32 on dist-test-slave-mjjr"
server_key: "66f546ca9374e58fc72107095314c011"
server_key_iv: "0e10a97b5246227d72359ee8ec92a0b0"
server_key_version: "encryptionkey@0"
I20260811 12:08:32.033886 12452 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:08:32.037784 12476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:08:32.039034 12452 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:08:32.039232 12452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "fabdffa403f743c594919bf8bb0ef113"
format_stamp: "Formatted at 2026-08-11 12:08:32 on dist-test-slave-mjjr"
server_key: "66f546ca9374e58fc72107095314c011"
server_key_iv: "0e10a97b5246227d72359ee8ec92a0b0"
server_key_version: "encryptionkey@0"
I20260811 12:08:32.039450 12452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:08:32.073021 12452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:08:32.077121 12452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:08:32.077433 12452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:08:32.096454 12452 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.190:45175
I20260811 12:08:32.096509 12559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.190:45175 every 8 connection(s)
I20260811 12:08:32.097859 12452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:08:32.101517 12561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:08:32.106851 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 12452
I20260811 12:08:32.107020 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:08:32.107558 10102 external_mini_cluster.cc:1428] Setting key 4cdf6ce0b95ecfa5ed0b2d23793eea3b
I20260811 12:08:32.112795 12561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fabdffa403f743c594919bf8bb0ef113: Bootstrap starting.
I20260811 12:08:32.116356 12561 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fabdffa403f743c594919bf8bb0ef113: Neither blocks nor log segments found. Creating new log.
I20260811 12:08:32.117532 12561 log.cc:824] T 00000000000000000000000000000000 P fabdffa403f743c594919bf8bb0ef113: Log is configured to *not* fsync() on all Append() calls
I20260811 12:08:32.120920 12561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fabdffa403f743c594919bf8bb0ef113: No bootstrap required, opened a new log
I20260811 12:08:32.121162 12564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:32.109410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33621 (local address 127.9.221.190:45175)
0811 12:08:32.110234 (+ 824us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:32.110245 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:32.110289 (+ 44us) server_negotiation.cc:407] Connection header received
0811 12:08:32.112023 (+ 1734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:32.112044 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:32.112428 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:32.112725 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:32.113364 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:32.114636 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:32.116563 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:32.117232 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:32.117366 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:32.118696 (+ 1330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:32.118710 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:32.118731 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:08:32.119476 (+ 745us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:32.119547 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:08:32.119571 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:32.119721 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:32.119921 (+ 200us) server_negotiation.cc:299] Negotiation successful
0811 12:08:32.120177 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":391,"threads_started":1}
I20260811 12:08:32.124938 10102 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:08:32.130482 12561 raft_consensus.cc:348] T 00000000000000000000000000000000 P fabdffa403f743c594919bf8bb0ef113 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fabdffa403f743c594919bf8bb0ef113" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 45175 } }
I20260811 12:08:32.130924 12561 raft_consensus.cc:374] T 00000000000000000000000000000000 P fabdffa403f743c594919bf8bb0ef113 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:08:32.131045 12561 raft_consensus.cc:729] T 00000000000000000000000000000000 P fabdffa403f743c594919bf8bb0ef113 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabdffa403f743c594919bf8bb0ef113, State: Initialized, Role: FOLLOWER
I20260811 12:08:32.131680 12561 consensus_queue.cc:260] T 00000000000000000000000000000000 P fabdffa403f743c594919bf8bb0ef113 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fabdffa403f743c594919bf8bb0ef113" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 45175 } }
I20260811 12:08:32.132014 12561 raft_consensus.cc:388] T 00000000000000000000000000000000 P fabdffa403f743c594919bf8bb0ef113 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:08:32.132169 12561 raft_consensus.cc:482] T 00000000000000000000000000000000 P fabdffa403f743c594919bf8bb0ef113 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:08:32.132364 12561 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fabdffa403f743c594919bf8bb0ef113 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:08:32.135411 12561 raft_consensus.cc:504] T 00000000000000000000000000000000 P fabdffa403f743c594919bf8bb0ef113 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fabdffa403f743c594919bf8bb0ef113" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 45175 } }
I20260811 12:08:32.136045 12564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:32.129308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51973 (local address 127.9.221.190:45175)
0811 12:08:32.129485 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:32.129491 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:32.129835 (+ 344us) server_negotiation.cc:407] Connection header received
0811 12:08:32.129985 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:32.129990 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:32.130064 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:32.130166 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:32.130650 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:32.131749 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:32.133247 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:32.133971 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:32.134112 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:32.135217 (+ 1105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:32.135224 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:32.135228 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:08:32.135250 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:32.135300 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:08:32.135337 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:32.135449 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:32.135598 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 12:08:32.135704 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 12:08:32.136044 12561 leader_election.cc:302] T 00000000000000000000000000000000 P fabdffa403f743c594919bf8bb0ef113 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fabdffa403f743c594919bf8bb0ef113; no voters:
I20260811 12:08:32.136762 12561 leader_election.cc:288] T 00000000000000000000000000000000 P fabdffa403f743c594919bf8bb0ef113 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:08:32.136883 12575 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fabdffa403f743c594919bf8bb0ef113 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:08:32.138162 12575 raft_consensus.cc:686] T 00000000000000000000000000000000 P fabdffa403f743c594919bf8bb0ef113 [term 1 LEADER]: Becoming Leader. State: Replica: fabdffa403f743c594919bf8bb0ef113, State: Running, Role: LEADER
I20260811 12:08:32.138655 12575 consensus_queue.cc:237] T 00000000000000000000000000000000 P fabdffa403f743c594919bf8bb0ef113 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fabdffa403f743c594919bf8bb0ef113" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 45175 } }
I20260811 12:08:32.138803 12561 sys_catalog.cc:564] T 00000000000000000000000000000000 P fabdffa403f743c594919bf8bb0ef113 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:08:32.141413 12580 sys_catalog.cc:455] T 00000000000000000000000000000000 P fabdffa403f743c594919bf8bb0ef113 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fabdffa403f743c594919bf8bb0ef113. Latest consensus state: current_term: 1 leader_uuid: "fabdffa403f743c594919bf8bb0ef113" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fabdffa403f743c594919bf8bb0ef113" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 45175 } } }
I20260811 12:08:32.141489 12576 sys_catalog.cc:455] T 00000000000000000000000000000000 P fabdffa403f743c594919bf8bb0ef113 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fabdffa403f743c594919bf8bb0ef113" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fabdffa403f743c594919bf8bb0ef113" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 45175 } } }
I20260811 12:08:32.141908 12576 sys_catalog.cc:458] T 00000000000000000000000000000000 P fabdffa403f743c594919bf8bb0ef113 [sys.catalog]: This master's current role is: LEADER
I20260811 12:08:32.141908 12580 sys_catalog.cc:458] T 00000000000000000000000000000000 P fabdffa403f743c594919bf8bb0ef113 [sys.catalog]: This master's current role is: LEADER
I20260811 12:08:32.148293 12586 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:08:32.153880 12586 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:08:32.162670 12586 catalog_manager.cc:1347] Generated new cluster ID: c5dfe2b2daf34243a571888d5d8a1e76
I20260811 12:08:32.162789 12586 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:08:32.171344 12586 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:08:32.172606 12586 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:08:32.186583 12586 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fabdffa403f743c594919bf8bb0ef113: Generated new TSK 0
I20260811 12:08:32.187330 12586 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:08:32.195801 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 12452
W20260811 12:08:32.197106 12549 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:33621
W20260811 12:08:32.197855 10102 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 12452: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:08:32.198024 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 12452
2026-08-11T12:08:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2223 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449743084027-10102-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:08:32 dist-test-slave-mjjr krb5kdc[12609](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:08:32 dist-test-slave-mjjr krb5kdc[12609](info): set up 2 sockets
Aug 11 12:08:32 dist-test-slave-mjjr krb5kdc[12609](info): commencing operation
krb5kdc: starting...
W20260811 12:08:33.528326 10102 mini_kdc.cc:121] Time spent starting KDC: real 1.285s user 0.002s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Aug 11 12:08:33 dist-test-slave-mjjr krb5kdc[12609](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:08:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:08:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.221.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
I20260811 12:08:33.829047 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:39617
--webserver_interface=127.9.221.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:45519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:39617
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:08:34.172960 12632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:08:34.173319 12632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:08:34.173498 12632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:08:34.181278 12632 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:08:34.181430 12632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:08:34.181535 12632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:08:34.181644 12632 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:08:34.181732 12632 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:08:34.191632 12632 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:45519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:39617
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:39617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.221.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:08:34.192973 12632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:08:34.194286 12632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:08:34.205941 12640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:34.205977 12645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:34.206022 12641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:08:34.207069 12632 server_base.cc:1034] running on GCE node
I20260811 12:08:34.207955 12632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:08:34.209388 12632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:08:34.210595 12632 hybrid_clock.cc:648] HybridClock initialized: now 1786450114210546 us; error 37 us; skew 500 ppm
I20260811 12:08:34.210940 12632 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:08:34 dist-test-slave-mjjr krb5kdc[12609](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450114, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:08:34.214099 12632 init.cc:377] Logged in from keytab as kudu/127.9.221.190@KRBTEST.COM (short username kudu)
I20260811 12:08:34.215498 12632 webserver.cc:460] Webserver started at http://127.9.221.190:44657/ using document root <none> and password file <none>
I20260811 12:08:34.216195 12632 fs_manager.cc:359] Metadata directory not provided
I20260811 12:08:34.216349 12632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:08:34.216672 12632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:08:34.219578 12632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "5c396ebf52d74ceab99c9146d3e60c1c"
format_stamp: "Formatted at 2026-08-11 12:08:34 on dist-test-slave-mjjr"
server_key: "369aa9d700d3845edbd294a3702deb57"
server_key_iv: "342befd86a5471b0b5444b6b983b1b26"
server_key_version: "encryptionkey@0"
I20260811 12:08:34.220422 12632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "5c396ebf52d74ceab99c9146d3e60c1c"
format_stamp: "Formatted at 2026-08-11 12:08:34 on dist-test-slave-mjjr"
server_key: "369aa9d700d3845edbd294a3702deb57"
server_key_iv: "342befd86a5471b0b5444b6b983b1b26"
server_key_version: "encryptionkey@0"
I20260811 12:08:34.225469 12632 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:08:34.228838 12658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:08:34.230057 12632 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:08:34.230249 12632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "5c396ebf52d74ceab99c9146d3e60c1c"
format_stamp: "Formatted at 2026-08-11 12:08:34 on dist-test-slave-mjjr"
server_key: "369aa9d700d3845edbd294a3702deb57"
server_key_iv: "342befd86a5471b0b5444b6b983b1b26"
server_key_version: "encryptionkey@0"
I20260811 12:08:34.230450 12632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:08:34.235951 12632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:08:34.239763 12632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:08:34.240046 12632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:08:34.258673 12632 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.190:39617
I20260811 12:08:34.258716 12733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.190:39617 every 8 connection(s)
I20260811 12:08:34.260119 12632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:08:34.260396 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 12632
I20260811 12:08:34.260581 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:08:34.261215 10102 external_mini_cluster.cc:1428] Setting key 1cb083fd2af9ae74f1f8be895a07c17d
I20260811 12:08:34.264532 12734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:08:34.268924 12739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:34.263100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41347 (local address 127.9.221.190:39617)
0811 12:08:34.263965 (+ 865us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:34.263975 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:34.264015 (+ 40us) server_negotiation.cc:407] Connection header received
0811 12:08:34.265642 (+ 1627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:34.265662 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:34.265983 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:34.266341 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:34.267403 (+ 1062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:34.267409 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:34.267421 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:08:34.267445 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:34.267515 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:08:34.267534 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:34.267775 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:34.267935 (+ 160us) server_negotiation.cc:299] Negotiation successful
0811 12:08:34.268105 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":646,"thread_start_us":371,"threads_started":1}
I20260811 12:08:34.272573 10102 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:08:34.275871 12734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c396ebf52d74ceab99c9146d3e60c1c: Bootstrap starting.
I20260811 12:08:34.279435 12739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:34.276831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47335 (local address 127.9.221.190:39617)
0811 12:08:34.277037 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:34.277046 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:34.277544 (+ 498us) server_negotiation.cc:407] Connection header received
0811 12:08:34.277708 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:34.277711 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:34.277759 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:34.277840 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:34.278765 (+ 925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:34.278769 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:34.278772 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:08:34.278786 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:34.278823 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:08:34.278837 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:34.278925 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:34.279081 (+ 156us) server_negotiation.cc:299] Negotiation successful
0811 12:08:34.279126 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 12:08:34.279791 12734 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c396ebf52d74ceab99c9146d3e60c1c: Neither blocks nor log segments found. Creating new log.
I20260811 12:08:34.280987 12734 log.cc:824] T 00000000000000000000000000000000 P 5c396ebf52d74ceab99c9146d3e60c1c: Log is configured to *not* fsync() on all Append() calls
I20260811 12:08:34.283860 12734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c396ebf52d74ceab99c9146d3e60c1c: No bootstrap required, opened a new log
I20260811 12:08:34.291565 12734 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c396ebf52d74ceab99c9146d3e60c1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c396ebf52d74ceab99c9146d3e60c1c" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 39617 } }
I20260811 12:08:34.292016 12734 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c396ebf52d74ceab99c9146d3e60c1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:08:34.292132 12734 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c396ebf52d74ceab99c9146d3e60c1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c396ebf52d74ceab99c9146d3e60c1c, State: Initialized, Role: FOLLOWER
I20260811 12:08:34.292691 12734 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c396ebf52d74ceab99c9146d3e60c1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c396ebf52d74ceab99c9146d3e60c1c" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 39617 } }
I20260811 12:08:34.292930 12734 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c396ebf52d74ceab99c9146d3e60c1c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:08:34.293083 12734 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c396ebf52d74ceab99c9146d3e60c1c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:08:34.293258 12734 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c396ebf52d74ceab99c9146d3e60c1c [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:08:34.295362 12734 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c396ebf52d74ceab99c9146d3e60c1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c396ebf52d74ceab99c9146d3e60c1c" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 39617 } }
I20260811 12:08:34.295921 12734 leader_election.cc:302] T 00000000000000000000000000000000 P 5c396ebf52d74ceab99c9146d3e60c1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c396ebf52d74ceab99c9146d3e60c1c; no voters:
I20260811 12:08:34.296495 12734 leader_election.cc:288] T 00000000000000000000000000000000 P 5c396ebf52d74ceab99c9146d3e60c1c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:08:34.296622 12750 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c396ebf52d74ceab99c9146d3e60c1c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:08:34.297773 12750 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c396ebf52d74ceab99c9146d3e60c1c [term 1 LEADER]: Becoming Leader. State: Replica: 5c396ebf52d74ceab99c9146d3e60c1c, State: Running, Role: LEADER
I20260811 12:08:34.298434 12750 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c396ebf52d74ceab99c9146d3e60c1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c396ebf52d74ceab99c9146d3e60c1c" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 39617 } }
I20260811 12:08:34.298632 12734 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c396ebf52d74ceab99c9146d3e60c1c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:08:34.301381 12752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c396ebf52d74ceab99c9146d3e60c1c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c396ebf52d74ceab99c9146d3e60c1c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c396ebf52d74ceab99c9146d3e60c1c" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 39617 } } }
I20260811 12:08:34.301354 12753 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c396ebf52d74ceab99c9146d3e60c1c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c396ebf52d74ceab99c9146d3e60c1c. Latest consensus state: current_term: 1 leader_uuid: "5c396ebf52d74ceab99c9146d3e60c1c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c396ebf52d74ceab99c9146d3e60c1c" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 39617 } } }
I20260811 12:08:34.301937 12753 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c396ebf52d74ceab99c9146d3e60c1c [sys.catalog]: This master's current role is: LEADER
I20260811 12:08:34.301942 12752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c396ebf52d74ceab99c9146d3e60c1c [sys.catalog]: This master's current role is: LEADER
I20260811 12:08:34.308207 12764 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:08:34.313517 12764 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:08:34.321375 12764 catalog_manager.cc:1347] Generated new cluster ID: 0e4defc91e724668a3dc9cd99e0011e6
I20260811 12:08:34.321488 12764 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:08:34.338200 12764 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:08:34.338719 12764 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:08:34.349805 12764 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c396ebf52d74ceab99c9146d3e60c1c: Generated new TSK 0
I20260811 12:08:34.350592 12764 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:08:34.365773 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 12632
W20260811 12:08:34.367527 12724 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:41347
W20260811 12:08:34.368268 10102 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 12632: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:08:34.368432 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 12632
2026-08-11T12:08:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2174 ms)
[----------] 18 tests from AuthTokenIssuingTest (40301 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:08:34 dist-test-slave-mjjr krb5kdc[12786](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:08:34 dist-test-slave-mjjr krb5kdc[12786](info): set up 2 sockets
Aug 11 12:08:34 dist-test-slave-mjjr krb5kdc[12786](info): commencing operation
krb5kdc: starting...
W20260811 12:08:35.701911 10102 mini_kdc.cc:121] Time spent starting KDC: real 1.283s user 0.000s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Aug 11 12:08:35 dist-test-slave-mjjr krb5kdc[12786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450115, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:08:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:08:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.221.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
I20260811 12:08:36.031318 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:45399
--webserver_interface=127.9.221.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:46553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:45399,127.9.221.189:42947,127.9.221.188:36605
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:08:36.377485 12804 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 12:08:36.377907 12804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:08:36.378043 12804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:08:36.378203 12804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:08:36.385179 12804 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:08:36.385299 12804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:08:36.385406 12804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:08:36.385516 12804 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:08:36.385608 12804 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:08:36.395107 12804 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:46553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:45399,127.9.221.189:42947,127.9.221.188:36605
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:45399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.221.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:08:36.396490 12804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:08:36.397873 12804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:08:36.410324 12814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:36.410280 12816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:36.410246 12818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:08:36.411065 12804 server_base.cc:1034] running on GCE node
I20260811 12:08:36.411875 12804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:08:36.413267 12804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:08:36.414513 12804 hybrid_clock.cc:648] HybridClock initialized: now 1786450116414439 us; error 51 us; skew 500 ppm
I20260811 12:08:36.414826 12804 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:08:36 dist-test-slave-mjjr krb5kdc[12786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450116, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:08:36.421594 12804 init.cc:377] Logged in from keytab as kudu/127.9.221.190@KRBTEST.COM (short username kudu)
I20260811 12:08:36.423056 12804 webserver.cc:460] Webserver started at http://127.9.221.190:43355/ using document root <none> and password file <none>
I20260811 12:08:36.423848 12804 fs_manager.cc:359] Metadata directory not provided
I20260811 12:08:36.424003 12804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:08:36.424325 12804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:08:36.427256 12804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "fdc0f75a2aa74bda9eb8e91589397794"
format_stamp: "Formatted at 2026-08-11 12:08:36 on dist-test-slave-mjjr"
server_key: "85219ade296348489824122af3c7f94d"
server_key_iv: "169d589b13b6e41907a6ed6048a31add"
server_key_version: "encryptionkey@0"
I20260811 12:08:36.428081 12804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "fdc0f75a2aa74bda9eb8e91589397794"
format_stamp: "Formatted at 2026-08-11 12:08:36 on dist-test-slave-mjjr"
server_key: "85219ade296348489824122af3c7f94d"
server_key_iv: "169d589b13b6e41907a6ed6048a31add"
server_key_version: "encryptionkey@0"
I20260811 12:08:36.433701 12804 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 12:08:36.437233 12832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:08:36.438628 12804 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 12:08:36.438817 12804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "fdc0f75a2aa74bda9eb8e91589397794"
format_stamp: "Formatted at 2026-08-11 12:08:36 on dist-test-slave-mjjr"
server_key: "85219ade296348489824122af3c7f94d"
server_key_iv: "169d589b13b6e41907a6ed6048a31add"
server_key_version: "encryptionkey@0"
I20260811 12:08:36.439045 12804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:08:36.469201 12804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:08:36.473400 12804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:08:36.473702 12804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:08:36.492980 12804 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.190:45399
I20260811 12:08:36.493000 12918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.190:45399 every 8 connection(s)
I20260811 12:08:36.494417 12804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:08:36.495098 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 12804
I20260811 12:08:36.495370 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:08:36.496201 10102 external_mini_cluster.cc:1428] Setting key af0bb0f403496262b20e3800d9edd367
I20260811 12:08:36.498500 12920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:08:36.505007 12920 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.221.190" port: 45399 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:08:36 dist-test-slave-mjjr krb5kdc[12786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450116, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:08:36.528000 12924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:36.508151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56798 (local address 127.9.221.190:45399)
0811 12:08:36.509845 (+ 1694us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:36.509851 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:36.509896 (+ 45us) server_negotiation.cc:407] Connection header received
0811 12:08:36.512123 (+ 2227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:36.512128 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:36.512327 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:36.512531 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:36.513780 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:36.514989 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:36.516445 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:36.516977 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:36.517172 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:36.520989 (+ 3817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:36.520996 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:36.521007 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:36.521040 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:36.523811 (+ 2771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:36.524736 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:36.524742 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:36.524744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:36.524878 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:36.525344 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:36.525348 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:36.525349 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:36.525786 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:36.525933 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:36.526642 (+ 709us) server_negotiation.cc:299] Negotiation successful
0811 12:08:36.526916 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":1506,"spinlock_wait_cycles":76544,"thread_start_us":1290,"threads_started":1}
I20260811 12:08:36.528055 12923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:36.508152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:45399 (local address 127.0.0.1:56798)
0811 12:08:36.509172 (+ 1020us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:36.509233 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 12:08:36.511782 (+ 2549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:08:36.512548 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:08:36.512555 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:08:36.513244 (+ 689us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:08:36.513567 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:08:36.513583 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:08:36.515190 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:08:36.515197 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:08:36.516301 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:08:36.516313 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:08:36.517204 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:08:36.517211 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:08:36.517414 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:36.518075 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:08:36.518096 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:08:36.520759 (+ 2663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:08:36.523939 (+ 3180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:08:36.523951 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:08:36.523963 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:08:36.524600 (+ 637us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:08:36.525001 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:08:36.525009 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:08:36.525015 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:08:36.525225 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:08:36.525937 (+ 712us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:08:36.525947 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:08:36.526218 (+ 271us) client_negotiation.cc:769] Sending connection context
0811 12:08:36.526547 (+ 329us) client_negotiation.cc:240] Negotiation successful
0811 12:08:36.527077 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":677,"mutex_wait_us":49,"thread_start_us":445,"threads_started":1}
WARNING: no policy specified for kudu/127.9.221.189@KRBTEST.COM; defaulting to no policy
I20260811 12:08:36.530761 12920 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42947 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.189@KRBTEST.COM" created.
I20260811 12:08:36.531873 12923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:36.531424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:42947 (local address 127.0.0.1:57278)
0811 12:08:36.531585 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:36.531752 (+ 167us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.189:42947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 12:08:36.532092 12833 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.221.189:42947: connect: Connection refused (error 111)
I20260811 12:08:36.533617 12923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:36.533270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:42947 (local address 127.0.0.1:57290)
0811 12:08:36.533435 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:36.533507 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.189:42947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 12:08:36.533931 12920 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.221.189:42947: Network error: Client connection negotiation failed: client connection to 127.9.221.189:42947: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.189.keytab.
Entry for principal kudu/127.9.221.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.189.keytab.
I20260811 12:08:36.566391 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.189:42947
--webserver_interface=127.9.221.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.189.keytab
--principal=kudu/127.9.221.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:46553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:45399,127.9.221.189:42947,127.9.221.188:36605
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:08:36.583338 12920 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42947 } attempt: 1
I20260811 12:08:36.584793 12923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:36.584144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:42947 (local address 127.0.0.1:57292)
0811 12:08:36.584404 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:36.584574 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.189:42947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260811 12:08:36.586158 12923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:36.585660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:42947 (local address 127.0.0.1:57298)
0811 12:08:36.585864 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:36.585966 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.189:42947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 12:08:36.586401 12920 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.221.189:42947: Network error: Client connection negotiation failed: client connection to 127.9.221.189:42947: connect: Connection refused (error 111)
I20260811 12:08:36.654709 12920 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42947 } attempt: 2
I20260811 12:08:36.656090 12923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:36.655459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:42947 (local address 127.0.0.1:57300)
0811 12:08:36.655768 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:36.655897 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.189:42947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 12:08:36.657177 12923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:36.656764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:42947 (local address 127.0.0.1:57312)
0811 12:08:36.656928 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:36.657026 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.189:42947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 12:08:36.657379 12920 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.221.189:42947: Network error: Client connection negotiation failed: client connection to 127.9.221.189:42947: connect: Connection refused (error 111)
I20260811 12:08:36.748659 12920 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42947 } attempt: 3
I20260811 12:08:36.749924 12923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:36.749481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:42947 (local address 127.0.0.1:57320)
0811 12:08:36.749696 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:36.749789 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.189:42947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260811 12:08:36.750973 12923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:36.750656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:42947 (local address 127.0.0.1:57328)
0811 12:08:36.750779 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:36.750846 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.189:42947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 12:08:36.751242 12920 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.221.189:42947: Network error: Client connection negotiation failed: client connection to 127.9.221.189:42947: connect: Connection refused (error 111)
I20260811 12:08:36.894589 12920 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42947 } attempt: 4
I20260811 12:08:36.895972 12923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:36.895323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:42947 (local address 127.0.0.1:57336)
0811 12:08:36.895747 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:36.895835 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.189:42947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217}
I20260811 12:08:36.896991 12923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:36.896615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:42947 (local address 127.0.0.1:57344)
0811 12:08:36.896777 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:36.896861 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.189:42947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260811 12:08:36.897228 12920 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.221.189:42947: Network error: Client connection negotiation failed: client connection to 127.9.221.189:42947: connect: Connection refused (error 111)
W20260811 12:08:36.908213 12934 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 12:08:36.908579 12934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:08:36.908680 12934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:08:36.908818 12934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:08:36.915838 12934 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:08:36.915908 12934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:08:36.916023 12934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:08:36.916077 12934 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:08:36.916182 12934 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:08:36.925539 12934 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:46553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:45399,127.9.221.189:42947,127.9.221.188:36605
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.189.keytab
--principal=kudu/127.9.221.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.189:42947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.221.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:08:36.926688 12934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:08:36.928049 12934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:08:36.939607 12945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:36.939692 12946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:36.939858 12948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:08:36.940840 12934 server_base.cc:1034] running on GCE node
I20260811 12:08:36.941548 12934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:08:36.942847 12934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:08:36.944069 12934 hybrid_clock.cc:648] HybridClock initialized: now 1786450116944018 us; error 34 us; skew 500 ppm
I20260811 12:08:36.944345 12934 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:08:36 dist-test-slave-mjjr krb5kdc[12786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450116, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:08:36.947395 12934 init.cc:377] Logged in from keytab as kudu/127.9.221.189@KRBTEST.COM (short username kudu)
I20260811 12:08:36.948876 12934 webserver.cc:460] Webserver started at http://127.9.221.189:32923/ using document root <none> and password file <none>
I20260811 12:08:36.949597 12934 fs_manager.cc:359] Metadata directory not provided
I20260811 12:08:36.949712 12934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:08:36.950042 12934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:08:36.952979 12934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-1/data/instance:
uuid: "2382c8a567d749d593c0047a4548d246"
format_stamp: "Formatted at 2026-08-11 12:08:36 on dist-test-slave-mjjr"
server_key: "f2eae3f171ba7a109f6b459b1fc67edb"
server_key_iv: "db21480dba91f05afacf7d79eedb2dc7"
server_key_version: "encryptionkey@0"
I20260811 12:08:36.953760 12934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-1/wal/instance:
uuid: "2382c8a567d749d593c0047a4548d246"
format_stamp: "Formatted at 2026-08-11 12:08:36 on dist-test-slave-mjjr"
server_key: "f2eae3f171ba7a109f6b459b1fc67edb"
server_key_iv: "db21480dba91f05afacf7d79eedb2dc7"
server_key_version: "encryptionkey@0"
I20260811 12:08:36.958813 12934 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.006s
I20260811 12:08:36.962172 12961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:08:36.963451 12934 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260811 12:08:36.963634 12934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-1/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-1/wal
uuid: "2382c8a567d749d593c0047a4548d246"
format_stamp: "Formatted at 2026-08-11 12:08:36 on dist-test-slave-mjjr"
server_key: "f2eae3f171ba7a109f6b459b1fc67edb"
server_key_iv: "db21480dba91f05afacf7d79eedb2dc7"
server_key_version: "encryptionkey@0"
I20260811 12:08:36.963902 12934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:08:36.976449 12934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:08:36.980152 12934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:08:36.980432 12934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:08:36.998687 12934 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.189:42947
I20260811 12:08:36.998723 13045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.189:42947 every 8 connection(s)
I20260811 12:08:37.000006 12934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-1/data/info.pb
I20260811 12:08:37.003494 13046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:08:37.008641 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 12934
I20260811 12:08:37.008841 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-1/wal/instance
I20260811 12:08:37.009265 10102 external_mini_cluster.cc:1428] Setting key d8c0c9db5b90503ab5416fb135ec54f1
I20260811 12:08:37.010043 13046 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.221.190" port: 45399 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:08:37 dist-test-slave-mjjr krb5kdc[12786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450116, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.189@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:08:37.033062 12924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:37.012524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56814 (local address 127.9.221.190:45399)
0811 12:08:37.012742 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:37.012748 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:37.013938 (+ 1190us) server_negotiation.cc:407] Connection header received
0811 12:08:37.016733 (+ 2795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:37.016737 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:37.016801 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:37.016894 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:37.018854 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:37.019834 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:37.022192 (+ 2358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:37.022867 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:37.022992 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:37.027047 (+ 4055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:37.027052 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:37.027056 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:37.027088 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:37.030078 (+ 2990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:37.030801 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:37.030806 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:37.030810 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:37.030934 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:37.031355 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:37.031360 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:37.031364 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:37.031724 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:37.031884 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:37.032523 (+ 639us) server_negotiation.cc:299] Negotiation successful
0811 12:08:37.032744 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 12:08:37.033949 13049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:37.012993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:45399 (local address 127.0.0.1:56814)
0811 12:08:37.013757 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:37.013812 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 12:08:37.016462 (+ 2650us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:08:37.017135 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:08:37.017147 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:08:37.017897 (+ 750us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:08:37.018651 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:08:37.018677 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:08:37.020005 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:08:37.020013 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:08:37.022031 (+ 2018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:08:37.022050 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:08:37.023099 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:08:37.023109 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:08:37.023425 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:37.024260 (+ 835us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:08:37.024293 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:08:37.026817 (+ 2524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:08:37.030224 (+ 3407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:08:37.030233 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:08:37.030246 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:08:37.030657 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:08:37.031049 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:08:37.031054 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:08:37.031058 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:08:37.031239 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:08:37.031859 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:08:37.031867 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:08:37.032168 (+ 301us) client_negotiation.cc:769] Sending connection context
0811 12:08:37.032416 (+ 248us) client_negotiation.cc:240] Negotiation successful
0811 12:08:37.032884 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":303,"threads_started":1}
I20260811 12:08:37.035655 13046 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42947 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.9.221.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.188@KRBTEST.COM" created.
Aug 11 12:08:37 dist-test-slave-mjjr krb5kdc[12786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450116, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.189@KRBTEST.COM for kudu/127.9.221.189@KRBTEST.COM
I20260811 12:08:37.051582 13049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:37.036272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:42947 (local address 127.0.0.1:57350)
0811 12:08:37.036475 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:37.036494 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:08:37.036584 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:08:37.037700 (+ 1116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:08:37.037704 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:08:37.038126 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:08:37.038266 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:08:37.038274 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:08:37.040379 (+ 2105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:08:37.040382 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:08:37.041152 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:08:37.041161 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:08:37.042405 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:08:37.042409 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:08:37.042570 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:37.043140 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:08:37.043156 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:08:37.045966 (+ 2810us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:08:37.049151 (+ 3185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:08:37.049159 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:08:37.049163 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:08:37.049518 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:08:37.049918 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:08:37.049923 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:08:37.049926 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:08:37.050063 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:08:37.050632 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:08:37.050642 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:08:37.050774 (+ 132us) client_negotiation.cc:769] Sending connection context
0811 12:08:37.050921 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 12:08:37.051187 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 12:08:37.051723 13054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:37.036548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57350 (local address 127.9.221.189:42947)
0811 12:08:37.037190 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:37.037200 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:37.037238 (+ 38us) server_negotiation.cc:407] Connection header received
0811 12:08:37.037292 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:37.037302 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:37.037543 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:37.037693 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:37.038456 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:37.040245 (+ 1789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:37.041295 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:37.042184 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:37.042331 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:37.046177 (+ 3846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:37.046188 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:37.046199 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:37.046229 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:37.049002 (+ 2773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:37.049649 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:37.049656 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:37.049660 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:37.049783 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:37.050177 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:37.050184 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:37.050187 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:37.050509 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:37.050717 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:37.051059 (+ 342us) server_negotiation.cc:299] Negotiation successful
0811 12:08:37.051302 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":451,"thread_start_us":304,"threads_started":1}
I20260811 12:08:37.054229 13046 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.221.188" port: 36605 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:08:37.055415 13049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:37.055006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:36605 (local address 127.0.0.1:52922)
0811 12:08:37.055170 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:37.055270 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:36605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 12:08:37.055794 12971 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.221.188:36605: connect: Connection refused (error 111)
I20260811 12:08:37.057308 13049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:37.056992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:36605 (local address 127.0.0.1:52928)
0811 12:08:37.057143 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:37.057220 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:36605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 12:08:37.057513 13046 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.221.188:36605: Network error: Client connection negotiation failed: client connection to 127.9.221.188:36605: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.188.keytab.
Entry for principal kudu/127.9.221.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.188.keytab.
I20260811 12:08:37.080912 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.188:36605
--webserver_interface=127.9.221.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.188.keytab
--principal=kudu/127.9.221.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:46553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:45399,127.9.221.189:42947,127.9.221.188:36605
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:08:37.106806 13046 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.221.188" port: 36605 } attempt: 1
I20260811 12:08:37.107959 13049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:37.107483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:36605 (local address 127.0.0.1:52944)
0811 12:08:37.107709 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:37.107804 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:36605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 12:08:37.109023 13049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:37.108684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:36605 (local address 127.0.0.1:52950)
0811 12:08:37.108852 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:37.108922 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:36605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 12:08:37.109218 13046 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.221.188:36605: Network error: Client connection negotiation failed: client connection to 127.9.221.188:36605: connect: Connection refused (error 111)
I20260811 12:08:37.177459 13046 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.221.188" port: 36605 } attempt: 2
I20260811 12:08:37.178617 13049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:37.178169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:36605 (local address 127.0.0.1:52956)
0811 12:08:37.178363 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:37.178465 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:36605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 12:08:37.179587 13049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:37.179255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:36605 (local address 127.0.0.1:52960)
0811 12:08:37.179415 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:37.179479 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:36605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 12:08:37.179811 13046 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.221.188:36605: Network error: Client connection negotiation failed: client connection to 127.9.221.188:36605: connect: Connection refused (error 111)
I20260811 12:08:37.196477 12920 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42947 } attempt: 5
Aug 11 12:08:37 dist-test-slave-mjjr krb5kdc[12786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450116, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for kudu/127.9.221.189@KRBTEST.COM
I20260811 12:08:37.210150 12923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:37.197144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:42947 (local address 127.0.0.1:57352)
0811 12:08:37.197348 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:37.197370 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:08:37.197504 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:08:37.197857 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:08:37.197860 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:08:37.198224 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:08:37.198407 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:08:37.198417 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:08:37.199822 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:08:37.199826 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:08:37.200677 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:08:37.200688 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:08:37.201650 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:08:37.201654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:08:37.201804 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:37.202459 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:08:37.202477 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:08:37.204492 (+ 2015us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:08:37.208026 (+ 3534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:08:37.208030 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:08:37.208033 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:08:37.208393 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:08:37.208770 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:08:37.208773 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:08:37.208775 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:08:37.208895 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:08:37.209481 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:08:37.209485 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:08:37.209606 (+ 121us) client_negotiation.cc:769] Sending connection context
0811 12:08:37.209721 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 12:08:37.209913 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260811 12:08:37.210389 13054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:37.197329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57352 (local address 127.9.221.189:42947)
0811 12:08:37.197536 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:37.197543 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:37.197565 (+ 22us) server_negotiation.cc:407] Connection header received
0811 12:08:37.197627 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:37.197632 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:37.197710 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:37.197825 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:37.198565 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:37.199700 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:37.200828 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:37.201505 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:37.201627 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:37.204680 (+ 3053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:37.204687 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:37.204691 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:37.204728 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:37.207903 (+ 3175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:37.208509 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:37.208514 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:37.208517 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:37.208663 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:37.208998 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:37.209003 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:37.209007 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:37.209369 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:37.209514 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:37.209850 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 12:08:37.210094 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 12:08:37.211495 12920 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.221.188" port: 36605 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:08:37.212517 12923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:37.212175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:36605 (local address 127.0.0.1:52974)
0811 12:08:37.212326 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:37.212399 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:36605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 12:08:37.213369 12923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:37.213065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:36605 (local address 127.0.0.1:52978)
0811 12:08:37.213227 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:37.213286 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:36605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 12:08:37.213583 12920 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.221.188:36605: Network error: Client connection negotiation failed: client connection to 127.9.221.188:36605: connect: Connection refused (error 111)
I20260811 12:08:37.264797 12920 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.221.188" port: 36605 } attempt: 1
I20260811 12:08:37.265997 12923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:37.265487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:36605 (local address 127.0.0.1:52992)
0811 12:08:37.265686 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:37.265784 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:36605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260811 12:08:37.267221 12923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:37.266841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:36605 (local address 127.0.0.1:52994)
0811 12:08:37.267008 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:37.267094 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:36605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 12:08:37.267442 12920 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.221.188:36605: Network error: Client connection negotiation failed: client connection to 127.9.221.188:36605: connect: Connection refused (error 111)
I20260811 12:08:37.271003 13046 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.221.188" port: 36605 } attempt: 3
I20260811 12:08:37.271960 13049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:37.271493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:36605 (local address 127.0.0.1:52998)
0811 12:08:37.271730 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:37.271814 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:36605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 12:08:37.273051 13049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:37.272648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:36605 (local address 127.0.0.1:53006)
0811 12:08:37.272888 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:37.272955 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:36605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
W20260811 12:08:37.273267 13046 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.221.188:36605: Network error: Client connection negotiation failed: client connection to 127.9.221.188:36605: connect: Connection refused (error 111)
I20260811 12:08:37.335713 12920 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.221.188" port: 36605 } attempt: 2
I20260811 12:08:37.336853 12923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:37.336364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:36605 (local address 127.0.0.1:53010)
0811 12:08:37.336598 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:37.336688 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:36605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260811 12:08:37.337859 12923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:37.337510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:36605 (local address 127.0.0.1:53018)
0811 12:08:37.337661 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:37.337746 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:36605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 12:08:37.338078 12920 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.221.188:36605: Network error: Client connection negotiation failed: client connection to 127.9.221.188:36605: connect: Connection refused (error 111)
I20260811 12:08:37.416484 13046 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.221.188" port: 36605 } attempt: 4
I20260811 12:08:37.417724 13049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:37.417196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:36605 (local address 127.0.0.1:53020)
0811 12:08:37.417433 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:37.417549 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:36605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 12:08:37.418787 13049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:37.418420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:36605 (local address 127.0.0.1:53036)
0811 12:08:37.418572 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:37.418660 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:36605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 12:08:37.418991 13046 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.221.188:36605: Network error: Client connection negotiation failed: client connection to 127.9.221.188:36605: connect: Connection refused (error 111)
W20260811 12:08:37.424746 13057 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 12:08:37.425109 13057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:08:37.425233 13057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:08:37.425376 13057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:08:37.432366 13057 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:08:37.432488 13057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:08:37.432595 13057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:08:37.432694 13057 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:08:37.432785 13057 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:08:37.442433 13057 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:46553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:45399,127.9.221.189:42947,127.9.221.188:36605
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.188.keytab
--principal=kudu/127.9.221.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.188:36605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.221.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:08:37.443871 13057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:08:37.444264 12920 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.221.188" port: 36605 } attempt: 3
I20260811 12:08:37.445223 13057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 12:08:37.445245 12923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:37.444798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:36605 (local address 127.0.0.1:53050)
0811 12:08:37.445012 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:37.445089 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:36605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 12:08:37.446233 12923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:37.445850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:36605 (local address 127.0.0.1:53060)
0811 12:08:37.446062 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:37.446130 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:36605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
W20260811 12:08:37.446456 12920 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.221.188:36605: Network error: Client connection negotiation failed: client connection to 127.9.221.188:36605: connect: Connection refused (error 111)
W20260811 12:08:37.456976 13066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:37.457012 13064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:37.456974 13069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:08:37.457775 13057 server_base.cc:1034] running on GCE node
I20260811 12:08:37.458515 13057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:08:37.459780 13057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:08:37.460989 13057 hybrid_clock.cc:648] HybridClock initialized: now 1786450117460923 us; error 48 us; skew 500 ppm
I20260811 12:08:37.461249 13057 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:08:37 dist-test-slave-mjjr krb5kdc[12786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450117, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:08:37.464208 13057 init.cc:377] Logged in from keytab as kudu/127.9.221.188@KRBTEST.COM (short username kudu)
I20260811 12:08:37.465524 13057 webserver.cc:460] Webserver started at http://127.9.221.188:36409/ using document root <none> and password file <none>
I20260811 12:08:37.466107 13057 fs_manager.cc:359] Metadata directory not provided
I20260811 12:08:37.466192 13057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:08:37.466455 13057 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:08:37.469575 13057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-2/data/instance:
uuid: "72ca830006224c42bc1e148faee79368"
format_stamp: "Formatted at 2026-08-11 12:08:37 on dist-test-slave-mjjr"
server_key: "2f8825c91554b65f346e54047a51fa83"
server_key_iv: "7ddaf0472fa34b57e4711e73d1564c08"
server_key_version: "encryptionkey@0"
I20260811 12:08:37.470285 13057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-2/wal/instance:
uuid: "72ca830006224c42bc1e148faee79368"
format_stamp: "Formatted at 2026-08-11 12:08:37 on dist-test-slave-mjjr"
server_key: "2f8825c91554b65f346e54047a51fa83"
server_key_iv: "7ddaf0472fa34b57e4711e73d1564c08"
server_key_version: "encryptionkey@0"
I20260811 12:08:37.478698 13057 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.001s sys 0.008s
I20260811 12:08:37.483497 13079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:08:37.485255 13057 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:08:37.485476 13057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-2/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-2/wal
uuid: "72ca830006224c42bc1e148faee79368"
format_stamp: "Formatted at 2026-08-11 12:08:37 on dist-test-slave-mjjr"
server_key: "2f8825c91554b65f346e54047a51fa83"
server_key_iv: "7ddaf0472fa34b57e4711e73d1564c08"
server_key_version: "encryptionkey@0"
I20260811 12:08:37.485769 13057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:08:37.519694 13057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:08:37.525514 13057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:08:37.525830 13057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:08:37.547796 13057 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.188:36605
I20260811 12:08:37.547798 13178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.188:36605 every 8 connection(s)
I20260811 12:08:37.549731 13057 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-2/data/info.pb
I20260811 12:08:37.551051 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 13057
I20260811 12:08:37.551258 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449743084027-10102-0/minicluster-data/master-2/wal/instance
I20260811 12:08:37.551821 10102 external_mini_cluster.cc:1428] Setting key 05a20fe33f7e9c751e447e2e507bd0a9
I20260811 12:08:37.554730 13180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:08:37.570830 13180 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.221.190" port: 45399 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:08:37 dist-test-slave-mjjr krb5kdc[12786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450115, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.188@KRBTEST.COM
I20260811 12:08:37.598520 13183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:37.559800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53064 (local address 127.9.221.188:36605)
0811 12:08:37.568312 (+ 8512us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:37.568331 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:37.568429 (+ 98us) server_negotiation.cc:407] Connection header received
0811 12:08:37.571530 (+ 3101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:37.571570 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:37.572212 (+ 642us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:37.572717 (+ 505us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:37.574571 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:37.577243 (+ 2672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:37.579879 (+ 2636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:37.581280 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:37.582354 (+ 1074us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:37.588709 (+ 6355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:37.588724 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:37.588753 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:37.588816 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:37.593567 (+ 4751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:37.594299 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:37.594307 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:37.594331 (+ 24us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:37.594561 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:37.595091 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:37.595097 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:37.595101 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:37.595788 (+ 687us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:37.596087 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:37.596634 (+ 547us) server_negotiation.cc:299] Negotiation successful
0811 12:08:37.597081 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8179,"thread_start_us":7894,"threads_started":1}
Aug 11 12:08:37 dist-test-slave-mjjr krb5kdc[12786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450117, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.188@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:08:37.608780 13185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:37.574446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56826 (local address 127.9.221.190:45399)
0811 12:08:37.590182 (+ 15736us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:37.590187 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:37.590204 (+ 17us) server_negotiation.cc:407] Connection header received
0811 12:08:37.590254 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:37.590257 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:37.590316 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:37.590456 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:37.592613 (+ 2157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:37.594039 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:37.595980 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:37.596612 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:37.596738 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:37.600669 (+ 3931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:37.600678 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:37.600685 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:37.600734 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:37.605328 (+ 4594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:37.606395 (+ 1067us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:37.606400 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:37.606403 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:37.606588 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:37.607070 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:37.607080 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:37.607083 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:37.607396 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:37.607581 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:37.608095 (+ 514us) server_negotiation.cc:299] Negotiation successful
0811 12:08:37.608311 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15573,"spinlock_wait_cycles":8064,"thread_start_us":15553,"threads_started":1}
I20260811 12:08:37.608767 13186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:37.574458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:45399 (local address 127.0.0.1:56826)
0811 12:08:37.589521 (+ 15063us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:37.589577 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 12:08:37.589762 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:08:37.590497 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:08:37.590506 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:08:37.591569 (+ 1063us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:08:37.592344 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:08:37.592359 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:08:37.594227 (+ 1868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:08:37.594234 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:08:37.595767 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:08:37.595780 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:08:37.596994 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:08:37.597001 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:08:37.597165 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:37.597898 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:08:37.597919 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:08:37.600418 (+ 2499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:08:37.605536 (+ 5118us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:08:37.605553 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:08:37.605564 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:08:37.606241 (+ 677us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:08:37.606711 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:08:37.606718 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:08:37.606724 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:08:37.606913 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:08:37.607549 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:08:37.607557 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:08:37.607789 (+ 232us) client_negotiation.cc:769] Sending connection context
0811 12:08:37.607984 (+ 195us) client_negotiation.cc:240] Negotiation successful
0811 12:08:37.608313 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14715,"thread_start_us":15412,"threads_started":1}
I20260811 12:08:37.610996 13180 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42947 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:08:37 dist-test-slave-mjjr krb5kdc[12786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450115, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
Aug 11 12:08:37 dist-test-slave-mjjr krb5kdc[12786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450117, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.188@KRBTEST.COM for kudu/127.9.221.189@KRBTEST.COM
I20260811 12:08:37.621903 13188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:37.602100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56834 (local address 127.9.221.190:45399)
0811 12:08:37.604228 (+ 2128us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:37.604232 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:37.604252 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:08:37.604292 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:37.604295 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:37.604348 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:37.604472 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:37.605993 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:37.607105 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:37.609278 (+ 2173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:37.609746 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:37.609903 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:37.615161 (+ 5258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:37.615169 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:37.615173 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:37.615215 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:37.618875 (+ 3660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:37.619788 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:37.619793 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:37.619797 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:37.619922 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:37.620303 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:37.620307 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:37.620311 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:37.620676 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:37.620952 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:37.621263 (+ 311us) server_negotiation.cc:299] Negotiation successful
0811 12:08:37.621485 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2000,"thread_start_us":1741,"threads_started":1}
I20260811 12:08:37.626374 12920 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.221.188" port: 36605 } attempt: 4
I20260811 12:08:37.629163 13054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:37.611917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57354 (local address 127.9.221.189:42947)
0811 12:08:37.612143 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:37.612151 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:37.612173 (+ 22us) server_negotiation.cc:407] Connection header received
0811 12:08:37.612240 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:37.612249 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:37.612336 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:37.612463 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:37.613413 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:37.614698 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:37.616567 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:37.617369 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:37.617538 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:37.621568 (+ 4030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:37.621576 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:37.621583 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:37.621631 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:37.625816 (+ 4185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:37.626685 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:37.626692 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:37.626697 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:37.626882 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:37.627405 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:37.627413 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:37.627419 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:37.627806 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:37.627937 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:37.628527 (+ 590us) server_negotiation.cc:299] Negotiation successful
0811 12:08:37.628744 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 12:08:37.629977 13186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:37.611795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:42947 (local address 127.0.0.1:57354)
0811 12:08:37.611961 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:37.611980 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:08:37.612099 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:08:37.612532 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:08:37.612538 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:08:37.613023 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:08:37.613223 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:08:37.613235 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:08:37.614882 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:08:37.614888 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:08:37.616395 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:08:37.616410 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:08:37.617554 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:08:37.617561 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:08:37.617761 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:37.618395 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:08:37.618419 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:08:37.621322 (+ 2903us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:08:37.625992 (+ 4670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:08:37.626000 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:08:37.626004 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:08:37.626534 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:08:37.627048 (+ 514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:08:37.627052 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:08:37.627055 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:08:37.627248 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:08:37.628012 (+ 764us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:08:37.628017 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:08:37.628216 (+ 199us) client_negotiation.cc:769] Sending connection context
0811 12:08:37.629351 (+ 1135us) client_negotiation.cc:240] Negotiation successful
0811 12:08:37.629631 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":22}
I20260811 12:08:37.631558 13180 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.221.188" port: 36605 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:08:37 dist-test-slave-mjjr krb5kdc[12786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450116, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for kudu/127.9.221.188@KRBTEST.COM
Aug 11 12:08:37 dist-test-slave-mjjr krb5kdc[12786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450115, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.189@KRBTEST.COM
I20260811 12:08:37.647718 12923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:37.626943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:36605 (local address 127.0.0.1:53074)
0811 12:08:37.627268 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:37.627287 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:08:37.627382 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:08:37.628224 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:08:37.628229 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:08:37.628722 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:08:37.628933 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:08:37.628946 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:08:37.630299 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:08:37.630306 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:08:37.631871 (+ 1565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:08:37.631885 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:08:37.633680 (+ 1795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:08:37.633707 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:08:37.634022 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:37.634722 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:08:37.634750 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:08:37.638599 (+ 3849us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:08:37.643889 (+ 5290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:08:37.643897 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:08:37.643904 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:08:37.644587 (+ 683us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:08:37.645125 (+ 538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:08:37.645133 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:08:37.645139 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:08:37.645349 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:08:37.646294 (+ 945us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:08:37.646302 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:08:37.646555 (+ 253us) client_negotiation.cc:769] Sending connection context
0811 12:08:37.646747 (+ 192us) client_negotiation.cc:240] Negotiation successful
0811 12:08:37.647135 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216}
I20260811 12:08:37.648645 13183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:37.627568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53074 (local address 127.9.221.188:36605)
0811 12:08:37.627760 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:37.627765 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:37.627784 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:08:37.627842 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:37.627849 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:37.627919 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:37.628001 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:37.629155 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:37.630165 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:37.632037 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:37.633296 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:37.633447 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:37.638885 (+ 5438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:37.638894 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:37.638901 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:37.638950 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:37.643694 (+ 4744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:37.644772 (+ 1078us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:37.644780 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:37.644786 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:37.644968 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:37.645504 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:37.645511 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:37.645517 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:37.645993 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:37.646162 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:37.648076 (+ 1914us) server_negotiation.cc:299] Negotiation successful
0811 12:08:37.648247 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":961,"server-negotiator.queue_time_us":93}
I20260811 12:08:37.649650 13190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:37.623867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57360 (local address 127.9.221.189:42947)
0811 12:08:37.626401 (+ 2534us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:37.626409 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:37.626438 (+ 29us) server_negotiation.cc:407] Connection header received
0811 12:08:37.626506 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:37.626513 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:37.626598 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:37.626725 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:37.629128 (+ 2403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:37.630606 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:37.632786 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:37.633608 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:37.633760 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:37.640363 (+ 6603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:37.640372 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:37.640377 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:37.640428 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:37.644703 (+ 4275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:37.645707 (+ 1004us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:37.645715 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:37.645720 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:37.645881 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:37.646378 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:37.646417 (+ 39us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:37.646423 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:37.646826 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:37.647123 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:37.648169 (+ 1046us) server_negotiation.cc:299] Negotiation successful
0811 12:08:37.648428 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2377,"thread_start_us":2171,"threads_started":1}
I20260811 12:08:37.654992 10102 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:08:37.655292 12920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdc0f75a2aa74bda9eb8e91589397794: Bootstrap starting.
Aug 11 12:08:37 dist-test-slave-mjjr krb5kdc[12786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450117, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.188@KRBTEST.COM for kudu/127.9.221.188@KRBTEST.COM
I20260811 12:08:37.659603 12920 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fdc0f75a2aa74bda9eb8e91589397794: Neither blocks nor log segments found. Creating new log.
I20260811 12:08:37.661242 12920 log.cc:824] T 00000000000000000000000000000000 P fdc0f75a2aa74bda9eb8e91589397794: Log is configured to *not* fsync() on all Append() calls
I20260811 12:08:37.658910 12870 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "fdc0f75a2aa74bda9eb8e91589397794"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56834
I20260811 12:08:37.665171 12920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdc0f75a2aa74bda9eb8e91589397794: No bootstrap required, opened a new log
I20260811 12:08:37.670164 13186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:37.632277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:36605 (local address 127.0.0.1:53084)
0811 12:08:37.632702 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:37.632720 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:08:37.632811 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:08:37.648400 (+ 15589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:08:37.648405 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:08:37.648939 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:08:37.649777 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:08:37.649791 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:08:37.652939 (+ 3148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:08:37.652947 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:08:37.654749 (+ 1802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:08:37.654767 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:08:37.656084 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:08:37.656092 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:08:37.656348 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:37.656987 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:08:37.657016 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:08:37.660221 (+ 3205us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:08:37.666533 (+ 6312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:08:37.666540 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:08:37.666546 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:08:37.667147 (+ 601us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:08:37.667750 (+ 603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:08:37.667758 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:08:37.667764 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:08:37.667944 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:08:37.668732 (+ 788us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:08:37.668740 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:08:37.668971 (+ 231us) client_negotiation.cc:769] Sending connection context
0811 12:08:37.669231 (+ 260us) client_negotiation.cc:240] Negotiation successful
0811 12:08:37.669653 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"mutex_wait_us":1}
I20260811 12:08:37.671072 13191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:37.632609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53084 (local address 127.9.221.188:36605)
0811 12:08:37.648077 (+ 15468us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:37.648082 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:37.648103 (+ 21us) server_negotiation.cc:407] Connection header received
0811 12:08:37.648147 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:37.648151 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:37.648222 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:37.648339 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:37.649942 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:37.652759 (+ 2817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:37.654919 (+ 2160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:37.655721 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:37.655831 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:37.660459 (+ 4628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:37.660469 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:37.660476 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:37.660518 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:37.666350 (+ 5832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:37.667276 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:37.667283 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:37.667287 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:37.667444 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:37.668075 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:37.668080 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:37.668083 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:37.668583 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:37.668720 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:37.669367 (+ 647us) server_negotiation.cc:299] Negotiation successful
0811 12:08:37.670560 (+ 1193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":914,"server-negotiator.queue_time_us":15338,"thread_start_us":15157,"threads_started":1}
I20260811 12:08:37.673502 12920 raft_consensus.cc:348] T 00000000000000000000000000000000 P fdc0f75a2aa74bda9eb8e91589397794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdc0f75a2aa74bda9eb8e91589397794" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 45399 } } peers { permanent_uuid: "2382c8a567d749d593c0047a4548d246" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42947 } } peers { permanent_uuid: "72ca830006224c42bc1e148faee79368" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 36605 } }
I20260811 12:08:37.673929 12920 raft_consensus.cc:729] T 00000000000000000000000000000000 P fdc0f75a2aa74bda9eb8e91589397794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdc0f75a2aa74bda9eb8e91589397794, State: Initialized, Role: FOLLOWER
I20260811 12:08:37.674451 12920 consensus_queue.cc:260] T 00000000000000000000000000000000 P fdc0f75a2aa74bda9eb8e91589397794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdc0f75a2aa74bda9eb8e91589397794" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 45399 } } peers { permanent_uuid: "2382c8a567d749d593c0047a4548d246" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42947 } } peers { permanent_uuid: "72ca830006224c42bc1e148faee79368" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 36605 } }
I20260811 12:08:37.675729 13195 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdc0f75a2aa74bda9eb8e91589397794 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdc0f75a2aa74bda9eb8e91589397794" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 45399 } } peers { permanent_uuid: "2382c8a567d749d593c0047a4548d246" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42947 } } peers { permanent_uuid: "72ca830006224c42bc1e148faee79368" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 36605 } } }
I20260811 12:08:37.676509 13195 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdc0f75a2aa74bda9eb8e91589397794 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:08:37.677009 12920 sys_catalog.cc:564] T 00000000000000000000000000000000 P fdc0f75a2aa74bda9eb8e91589397794 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:08:37.677081 12870 raft_consensus.cc:482] T 00000000000000000000000000000000 P fdc0f75a2aa74bda9eb8e91589397794 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 12:08:37.677259 12870 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fdc0f75a2aa74bda9eb8e91589397794 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:08:37.677469 13180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72ca830006224c42bc1e148faee79368: Bootstrap starting.
I20260811 12:08:37.681542 12870 raft_consensus.cc:504] T 00000000000000000000000000000000 P fdc0f75a2aa74bda9eb8e91589397794 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdc0f75a2aa74bda9eb8e91589397794" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 45399 } } peers { permanent_uuid: "2382c8a567d749d593c0047a4548d246" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42947 } } peers { permanent_uuid: "72ca830006224c42bc1e148faee79368" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 36605 } }
I20260811 12:08:37.681831 13180 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72ca830006224c42bc1e148faee79368: Neither blocks nor log segments found. Creating new log.
I20260811 12:08:37.683243 13180 log.cc:824] T 00000000000000000000000000000000 P 72ca830006224c42bc1e148faee79368: Log is configured to *not* fsync() on all Append() calls
I20260811 12:08:37.685853 12870 leader_election.cc:288] T 00000000000000000000000000000000 P fdc0f75a2aa74bda9eb8e91589397794 [CANDIDATE]: Term 1 election: Requested vote from peers 2382c8a567d749d593c0047a4548d246 (127.9.221.189:42947), 72ca830006224c42bc1e148faee79368 (127.9.221.188:36605)
I20260811 12:08:37.698868 13180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72ca830006224c42bc1e148faee79368: No bootstrap required, opened a new log
I20260811 12:08:37.712749 13180 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72ca830006224c42bc1e148faee79368 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdc0f75a2aa74bda9eb8e91589397794" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 45399 } } peers { permanent_uuid: "2382c8a567d749d593c0047a4548d246" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42947 } } peers { permanent_uuid: "72ca830006224c42bc1e148faee79368" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 36605 } }
I20260811 12:08:37.713292 13180 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72ca830006224c42bc1e148faee79368 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72ca830006224c42bc1e148faee79368, State: Initialized, Role: FOLLOWER
I20260811 12:08:37.713867 13180 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72ca830006224c42bc1e148faee79368 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdc0f75a2aa74bda9eb8e91589397794" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 45399 } } peers { permanent_uuid: "2382c8a567d749d593c0047a4548d246" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42947 } } peers { permanent_uuid: "72ca830006224c42bc1e148faee79368" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 36605 } }
I20260811 12:08:37.718216 13046 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.221.188" port: 36605 } attempt: 5
W20260811 12:08:37.726661 12840 tablet.cc:2369] T 00000000000000000000000000000000 P fdc0f75a2aa74bda9eb8e91589397794: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 12:08:37 dist-test-slave-mjjr krb5kdc[12786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450116, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.189@KRBTEST.COM for kudu/127.9.221.188@KRBTEST.COM
I20260811 12:08:37.739776 13049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:37.719010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:36605 (local address 127.0.0.1:53090)
0811 12:08:37.719483 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:37.719507 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 12:08:37.719631 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:08:37.720234 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:08:37.720242 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:08:37.720887 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:08:37.721146 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:08:37.721161 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:08:37.722873 (+ 1712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:08:37.722881 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:08:37.724757 (+ 1876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:08:37.724778 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:08:37.726121 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:08:37.726131 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:08:37.726503 (+ 372us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:37.727294 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:08:37.727325 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:08:37.730862 (+ 3537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:08:37.736189 (+ 5327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:08:37.736197 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:08:37.736203 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:08:37.736858 (+ 655us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:08:37.737396 (+ 538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:08:37.737404 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:08:37.737410 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:08:37.737593 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:08:37.738428 (+ 835us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:08:37.738436 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:08:37.738630 (+ 194us) client_negotiation.cc:769] Sending connection context
0811 12:08:37.738834 (+ 204us) client_negotiation.cc:240] Negotiation successful
0811 12:08:37.739211 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298}
I20260811 12:08:37.741075 13191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:37.719468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53090 (local address 127.9.221.188:36605)
0811 12:08:37.719666 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:37.719672 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:37.719692 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:08:37.719783 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:37.719791 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:37.719883 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:37.719990 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:37.721349 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:37.722661 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:37.724956 (+ 2295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:37.725789 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:37.725920 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:37.731107 (+ 5187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:37.731116 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:37.731123 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:37.731173 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:37.735971 (+ 4798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:37.737031 (+ 1060us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:37.737039 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:37.737045 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:37.737234 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:37.737737 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:37.737744 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:37.737750 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:37.738180 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:37.738344 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:37.740311 (+ 1967us) server_negotiation.cc:299] Negotiation successful
0811 12:08:37.740611 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 12:08:37.742944 13220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72ca830006224c42bc1e148faee79368 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdc0f75a2aa74bda9eb8e91589397794" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 45399 } } peers { permanent_uuid: "2382c8a567d749d593c0047a4548d246" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42947 } } peers { permanent_uuid: "72ca830006224c42bc1e148faee79368" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 36605 } } }
I20260811 12:08:37.745653 13220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72ca830006224c42bc1e148faee79368 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:08:37.745504 13180 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72ca830006224c42bc1e148faee79368 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:08:37.747882 13137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fdc0f75a2aa74bda9eb8e91589397794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "72ca830006224c42bc1e148faee79368"
I20260811 12:08:37.748474 13137 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72ca830006224c42bc1e148faee79368 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:08:37.752449 13137 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 72ca830006224c42bc1e148faee79368 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fdc0f75a2aa74bda9eb8e91589397794 in term 1.
I20260811 12:08:37.753588 12836 leader_election.cc:302] T 00000000000000000000000000000000 P fdc0f75a2aa74bda9eb8e91589397794 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72ca830006224c42bc1e148faee79368, fdc0f75a2aa74bda9eb8e91589397794; no voters:
I20260811 12:08:37.754211 13195 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fdc0f75a2aa74bda9eb8e91589397794 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:08:37.756258 13046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2382c8a567d749d593c0047a4548d246: Bootstrap starting.
I20260811 12:08:37.772343 13188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:37.725074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56838 (local address 127.9.221.190:45399)
0811 12:08:37.725371 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:37.725381 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:37.738242 (+ 12861us) server_negotiation.cc:407] Connection header received
0811 12:08:37.740924 (+ 2682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:37.740932 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:37.741027 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:37.741111 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:37.749418 (+ 8307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:37.750555 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:37.756399 (+ 5844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:37.757108 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:37.757213 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:37.763325 (+ 6112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:37.763333 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:37.763340 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:37.763389 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:37.767508 (+ 4119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:37.768910 (+ 1402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:37.768915 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:37.768919 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:37.769100 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:37.769896 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:37.769901 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:37.769904 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:37.770286 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:37.770536 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:37.771624 (+ 1088us) server_negotiation.cc:299] Negotiation successful
0811 12:08:37.771922 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260811 12:08:37.774289 13195 raft_consensus.cc:686] T 00000000000000000000000000000000 P fdc0f75a2aa74bda9eb8e91589397794 [term 1 LEADER]: Becoming Leader. State: Replica: fdc0f75a2aa74bda9eb8e91589397794, State: Running, Role: LEADER
I20260811 12:08:37.775069 13195 consensus_queue.cc:237] T 00000000000000000000000000000000 P fdc0f75a2aa74bda9eb8e91589397794 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdc0f75a2aa74bda9eb8e91589397794" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 45399 } } peers { permanent_uuid: "2382c8a567d749d593c0047a4548d246" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42947 } } peers { permanent_uuid: "72ca830006224c42bc1e148faee79368" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 36605 } }
W20260811 12:08:37.778409 13089 tablet.cc:2369] T 00000000000000000000000000000000 P 72ca830006224c42bc1e148faee79368: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:08:37.779740 13046 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2382c8a567d749d593c0047a4548d246: Neither blocks nor log segments found. Creating new log.
I20260811 12:08:37.787719 13235 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdc0f75a2aa74bda9eb8e91589397794 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fdc0f75a2aa74bda9eb8e91589397794. Latest consensus state: current_term: 1 leader_uuid: "fdc0f75a2aa74bda9eb8e91589397794" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdc0f75a2aa74bda9eb8e91589397794" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 45399 } } peers { permanent_uuid: "2382c8a567d749d593c0047a4548d246" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42947 } } peers { permanent_uuid: "72ca830006224c42bc1e148faee79368" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 36605 } } }
I20260811 12:08:37.788455 13235 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdc0f75a2aa74bda9eb8e91589397794 [sys.catalog]: This master's current role is: LEADER
I20260811 12:08:37.789911 13046 log.cc:824] T 00000000000000000000000000000000 P 2382c8a567d749d593c0047a4548d246: Log is configured to *not* fsync() on all Append() calls
I20260811 12:08:37.798477 13253 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:08:37.799345 13251 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 12:08:37.807123 12804 master_runner.cc:186] Error getting master registration for 127.9.221.189:42947: OK, instance_id { permanent_uuid: "2382c8a567d749d593c0047a4548d246" instance_seqno: 1786450116993493 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 12:08:37.825781 13046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2382c8a567d749d593c0047a4548d246: No bootstrap required, opened a new log
I20260811 12:08:37.830783 12804 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 12:08:37.833817 13259 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 72ca830006224c42bc1e148faee79368: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 12:08:37.834046 13259 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 12:08:37.830545 13253 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:08:37.851266 13183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:37.726004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53098 (local address 127.9.221.188:36605)
0811 12:08:37.726177 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:37.726183 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:37.741263 (+ 15080us) server_negotiation.cc:407] Connection header received
0811 12:08:37.760412 (+ 19149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:37.760417 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:37.760523 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:37.760610 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:37.772133 (+ 11523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:37.773252 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:37.788364 (+ 15112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:37.789290 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:37.789429 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:37.810706 (+ 21277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:37.810715 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:37.810723 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:37.810774 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:37.815287 (+ 4513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:37.828760 (+ 13473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:37.828769 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:37.828772 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:37.828947 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:37.845048 (+ 16101us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:37.845058 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:37.845064 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:37.845584 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:37.845894 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:37.850399 (+ 4505us) server_negotiation.cc:299] Negotiation successful
0811 12:08:37.850736 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 12:08:37.854844 13190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:37.738930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57370 (local address 127.9.221.189:42947)
0811 12:08:37.739156 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:37.739164 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:37.762406 (+ 23242us) server_negotiation.cc:407] Connection header received
0811 12:08:37.763512 (+ 1106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:37.763521 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:37.763607 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:37.763709 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:37.774937 (+ 11228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:37.776160 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:37.790959 (+ 14799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:37.791872 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:37.792011 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:37.805618 (+ 13607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:37.805629 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:37.805636 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:37.805692 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:37.810340 (+ 4648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:37.828327 (+ 17987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:37.828336 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:37.828342 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:37.828536 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:37.846048 (+ 17512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:37.846058 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:37.846064 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:37.846547 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:37.846835 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:37.854050 (+ 7215us) server_negotiation.cc:299] Negotiation successful
0811 12:08:37.854386 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 12:08:37.834290 13057 master_runner.cc:186] Error getting master registration for 127.9.221.189:42947: OK, instance_id { permanent_uuid: "2382c8a567d749d593c0047a4548d246" instance_seqno: 1786450116993493 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 12:08:37.858187 13057 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 12:08:37.874404 13046 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2382c8a567d749d593c0047a4548d246 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdc0f75a2aa74bda9eb8e91589397794" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 45399 } } peers { permanent_uuid: "2382c8a567d749d593c0047a4548d246" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42947 } } peers { permanent_uuid: "72ca830006224c42bc1e148faee79368" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 36605 } }
I20260811 12:08:37.874840 13046 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2382c8a567d749d593c0047a4548d246 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2382c8a567d749d593c0047a4548d246, State: Initialized, Role: FOLLOWER
I20260811 12:08:37.875346 13046 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2382c8a567d749d593c0047a4548d246 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdc0f75a2aa74bda9eb8e91589397794" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 45399 } } peers { permanent_uuid: "2382c8a567d749d593c0047a4548d246" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42947 } } peers { permanent_uuid: "72ca830006224c42bc1e148faee79368" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 36605 } }
I20260811 12:08:37.878121 13137 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 72ca830006224c42bc1e148faee79368 [term 1 FOLLOWER]: Refusing update from remote peer fdc0f75a2aa74bda9eb8e91589397794: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:08:37.880141 13235 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fdc0f75a2aa74bda9eb8e91589397794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72ca830006224c42bc1e148faee79368" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 36605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:08:37.889917 13220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72ca830006224c42bc1e148faee79368 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fdc0f75a2aa74bda9eb8e91589397794. Latest consensus state: current_term: 1 leader_uuid: "fdc0f75a2aa74bda9eb8e91589397794" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdc0f75a2aa74bda9eb8e91589397794" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 45399 } } peers { permanent_uuid: "2382c8a567d749d593c0047a4548d246" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42947 } } peers { permanent_uuid: "72ca830006224c42bc1e148faee79368" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 36605 } } }
I20260811 12:08:37.890165 13220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72ca830006224c42bc1e148faee79368 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:08:37.892246 13262 mvcc.cc:205] Tried to move back new op lower bound from 7317299682787127296 to 7317299682414522368. Current Snapshot: MvccSnapshot[applied={T|T < 7317299682787127296}]
I20260811 12:08:37.892853 13195 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdc0f75a2aa74bda9eb8e91589397794 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fdc0f75a2aa74bda9eb8e91589397794" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdc0f75a2aa74bda9eb8e91589397794" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 45399 } } peers { permanent_uuid: "2382c8a567d749d593c0047a4548d246" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42947 } } peers { permanent_uuid: "72ca830006224c42bc1e148faee79368" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 36605 } } }
I20260811 12:08:37.893164 13195 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdc0f75a2aa74bda9eb8e91589397794 [sys.catalog]: This master's current role is: LEADER
I20260811 12:08:37.897502 13253 catalog_manager.cc:1347] Generated new cluster ID: 9e15ed8a7a454f40837501f19978cb47
I20260811 12:08:37.897612 13253 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:08:37.900128 13266 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2382c8a567d749d593c0047a4548d246 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdc0f75a2aa74bda9eb8e91589397794" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 45399 } } peers { permanent_uuid: "2382c8a567d749d593c0047a4548d246" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42947 } } peers { permanent_uuid: "72ca830006224c42bc1e148faee79368" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 36605 } } }
I20260811 12:08:37.900552 13266 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2382c8a567d749d593c0047a4548d246 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:08:37.905342 13046 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2382c8a567d749d593c0047a4548d246 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:08:37.905454 13000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fdc0f75a2aa74bda9eb8e91589397794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "2382c8a567d749d593c0047a4548d246"
I20260811 12:08:37.905815 12999 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2382c8a567d749d593c0047a4548d246 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:08:37.909617 12999 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2382c8a567d749d593c0047a4548d246 [term 1 FOLLOWER]: Refusing update from remote peer fdc0f75a2aa74bda9eb8e91589397794: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:08:37.912863 13000 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2382c8a567d749d593c0047a4548d246 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fdc0f75a2aa74bda9eb8e91589397794 in term 1.
I20260811 12:08:37.915237 13235 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fdc0f75a2aa74bda9eb8e91589397794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2382c8a567d749d593c0047a4548d246" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:08:37.919922 13220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72ca830006224c42bc1e148faee79368 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fdc0f75a2aa74bda9eb8e91589397794" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdc0f75a2aa74bda9eb8e91589397794" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 45399 } } peers { permanent_uuid: "2382c8a567d749d593c0047a4548d246" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42947 } } peers { permanent_uuid: "72ca830006224c42bc1e148faee79368" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 36605 } } }
I20260811 12:08:37.920168 13220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72ca830006224c42bc1e148faee79368 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:08:37.942392 13253 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:08:37.943284 13274 mvcc.cc:205] Tried to move back new op lower bound from 7317299682787127296 to 7317299682414522368. Current Snapshot: MvccSnapshot[applied={T|T < 7317299682787127296}]
I20260811 12:08:37.944581 13253 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:08:37.956669 13266 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2382c8a567d749d593c0047a4548d246 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fdc0f75a2aa74bda9eb8e91589397794. Latest consensus state: current_term: 1 leader_uuid: "fdc0f75a2aa74bda9eb8e91589397794" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdc0f75a2aa74bda9eb8e91589397794" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 45399 } } peers { permanent_uuid: "2382c8a567d749d593c0047a4548d246" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42947 } } peers { permanent_uuid: "72ca830006224c42bc1e148faee79368" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 36605 } } }
I20260811 12:08:37.957157 13266 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2382c8a567d749d593c0047a4548d246 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:08:37.959163 13195 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdc0f75a2aa74bda9eb8e91589397794 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fdc0f75a2aa74bda9eb8e91589397794" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdc0f75a2aa74bda9eb8e91589397794" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 45399 } } peers { permanent_uuid: "2382c8a567d749d593c0047a4548d246" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42947 } } peers { permanent_uuid: "72ca830006224c42bc1e148faee79368" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 36605 } } }
I20260811 12:08:37.959755 13195 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdc0f75a2aa74bda9eb8e91589397794 [sys.catalog]: This master's current role is: LEADER
I20260811 12:08:37.968009 13278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2382c8a567d749d593c0047a4548d246 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fdc0f75a2aa74bda9eb8e91589397794" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdc0f75a2aa74bda9eb8e91589397794" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 45399 } } peers { permanent_uuid: "2382c8a567d749d593c0047a4548d246" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 42947 } } peers { permanent_uuid: "72ca830006224c42bc1e148faee79368" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 36605 } } }
I20260811 12:08:37.968571 13278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2382c8a567d749d593c0047a4548d246 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:08:37.969215 13253 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fdc0f75a2aa74bda9eb8e91589397794: Generated new TSK 0
I20260811 12:08:37.970088 13253 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:08:38.023165 13292 catalog_manager.cc:1259] Loaded cluster ID: 9e15ed8a7a454f40837501f19978cb47
I20260811 12:08:38.023372 13292 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2382c8a567d749d593c0047a4548d246: loading cluster ID for follower catalog manager: success
I20260811 12:08:38.027433 13292 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2382c8a567d749d593c0047a4548d246: acquiring CA information for follower catalog manager: success
I20260811 12:08:38.030124 13292 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2382c8a567d749d593c0047a4548d246: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 12:08:38.081091 10102 mini_kdc.cc:348] Time spent kdestroy: real 0.103s user 0.000s sys 0.032s
W20260811 12:08:38.135341 13188 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:08:38.117511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56848 (local address 127.9.221.190:45399)
0811 12:08:38.117711 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:38.117717 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:38.131492 (+ 13775us) server_negotiation.cc:407] Connection header received
0811 12:08:38.133189 (+ 1697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:38.133195 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:38.133291 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:38.133382 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:38.135030 (+ 1648us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56848: BlockingRecv error: recv got EOF from 127.0.0.1:56848 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260811 12:08:38.136991 13183 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:08:38.125786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53102 (local address 127.9.221.188:36605)
0811 12:08:38.125981 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:38.125987 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:38.132017 (+ 6030us) server_negotiation.cc:407] Connection header received
0811 12:08:38.133629 (+ 1612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:38.133635 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:38.133739 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:38.133810 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:38.136711 (+ 2901us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53102: BlockingRecv error: recv got EOF from 127.0.0.1:53102 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260811 12:08:38.140457 13190 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:08:38.132744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57382 (local address 127.9.221.189:42947)
0811 12:08:38.132935 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:38.132942 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:38.138442 (+ 5500us) server_negotiation.cc:407] Connection header received
0811 12:08:38.138622 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:38.138629 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:38.138739 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:38.138856 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:38.140180 (+ 1324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57382: BlockingRecv error: recv got EOF from 127.0.0.1:57382 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 12:08:38.190424 13190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:38.170541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57398 (local address 127.9.221.189:42947)
0811 12:08:38.170779 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:38.170787 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:38.182704 (+ 11917us) server_negotiation.cc:407] Connection header received
0811 12:08:38.182995 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:38.183005 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:38.183243 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:38.183487 (+ 244us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:08:38.184074 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:38.185246 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:38.187330 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:38.188138 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:38.188298 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:38.188912 (+ 614us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:08:38.189629 (+ 717us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:08:38.189822 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:38.189898 (+ 76us) server_negotiation.cc:299] Negotiation successful
0811 12:08:38.189997 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260811 12:08:38.236116 13183 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:08:38.226999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53110 (local address 127.9.221.188:36605)
0811 12:08:38.227191 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:38.227198 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:38.234353 (+ 7155us) server_negotiation.cc:407] Connection header received
0811 12:08:38.234629 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:38.234635 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:38.234777 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:38.234897 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:38.235825 (+ 928us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53110: BlockingRecv error: recv got EOF from 127.0.0.1:53110 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 12:08:38.287760 13190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:38.269089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57402 (local address 127.9.221.189:42947)
0811 12:08:38.269409 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:38.269419 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:38.281514 (+ 12095us) server_negotiation.cc:407] Connection header received
0811 12:08:38.281788 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:38.281795 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:38.281899 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:38.282023 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:08:38.282542 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:38.283350 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:38.285515 (+ 2165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:38.286071 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:38.286275 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:38.286800 (+ 525us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:08:38.286939 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:08:38.287060 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:38.287230 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 12:08:38.287325 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
W20260811 12:08:38.334049 13183 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:08:38.318267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53122 (local address 127.9.221.188:36605)
0811 12:08:38.318708 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:38.318715 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:38.332279 (+ 13564us) server_negotiation.cc:407] Connection header received
0811 12:08:38.332513 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:38.332519 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:38.332623 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:38.332738 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:38.333797 (+ 1059us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53122: BlockingRecv error: recv got EOF from 127.0.0.1:53122 (error 108)
Metrics: {"server-negotiator.queue_time_us":281}
I20260811 12:08:38.391095 13190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:38.372599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57412 (local address 127.9.221.189:42947)
0811 12:08:38.372830 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:38.372842 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:38.384903 (+ 12061us) server_negotiation.cc:407] Connection header received
0811 12:08:38.385202 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:38.385208 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:38.385320 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:38.385438 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:08:38.386038 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:38.386886 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:38.389029 (+ 2143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:38.389489 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:38.389678 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:38.390181 (+ 503us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:08:38.390310 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:08:38.390429 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:38.390586 (+ 157us) server_negotiation.cc:299] Negotiation successful
0811 12:08:38.390680 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260811 12:08:38.434005 13183 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:08:38.425142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53136 (local address 127.9.221.188:36605)
0811 12:08:38.425344 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:38.425350 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:38.432237 (+ 6887us) server_negotiation.cc:407] Connection header received
0811 12:08:38.432511 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:38.432518 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:38.432617 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:38.432740 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:38.433717 (+ 977us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53136: BlockingRecv error: recv got EOF from 127.0.0.1:53136 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 12:08:38.498445 13190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:38.478663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57418 (local address 127.9.221.189:42947)
0811 12:08:38.478924 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:38.478940 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:38.491602 (+ 12662us) server_negotiation.cc:407] Connection header received
0811 12:08:38.492523 (+ 921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:38.492530 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:38.492649 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:38.492805 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:08:38.493318 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:38.494166 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:38.496301 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:38.496810 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:38.496979 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:38.497487 (+ 508us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:08:38.497639 (+ 152us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:08:38.497758 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:38.497945 (+ 187us) server_negotiation.cc:299] Negotiation successful
0811 12:08:38.498040 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
W20260811 12:08:38.530153 13183 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:08:38.523593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53146 (local address 127.9.221.188:36605)
0811 12:08:38.523950 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:38.523958 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:38.528373 (+ 4415us) server_negotiation.cc:407] Connection header received
0811 12:08:38.528591 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:38.528597 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:38.528708 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:38.528853 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:38.529861 (+ 1008us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53146: BlockingRecv error: recv got EOF from 127.0.0.1:53146 (error 108)
Metrics: {"server-negotiator.queue_time_us":200}
I20260811 12:08:38.554555 13190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:38.541663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57424 (local address 127.9.221.189:42947)
0811 12:08:38.541878 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:38.541885 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:38.542450 (+ 565us) server_negotiation.cc:407] Connection header received
0811 12:08:38.542672 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:38.542678 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:38.542779 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:38.542913 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:08:38.543473 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:38.544795 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:38.551834 (+ 7039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:38.552656 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:38.552833 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:38.553553 (+ 720us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:08:38.553733 (+ 180us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:08:38.553805 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:38.554072 (+ 267us) server_negotiation.cc:299] Negotiation successful
0811 12:08:38.554184 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
W20260811 12:08:38.564815 13183 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:08:38.561957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53160 (local address 127.9.221.188:36605)
0811 12:08:38.562231 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:38.562241 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:38.563254 (+ 1013us) server_negotiation.cc:407] Connection header received
0811 12:08:38.563500 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:38.563512 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:38.563604 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:38.563752 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:38.564545 (+ 793us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53160: BlockingRecv error: recv got EOF from 127.0.0.1:53160 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 12:08:38.590353 13190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:38.584142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57426 (local address 127.9.221.189:42947)
0811 12:08:38.584390 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:38.584400 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:38.584975 (+ 575us) server_negotiation.cc:407] Connection header received
0811 12:08:38.585148 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:38.585153 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:38.585244 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:38.585369 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:08:38.585757 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:38.586913 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:38.588351 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:38.589064 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:38.589201 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:38.589595 (+ 394us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:08:38.589735 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:08:38.589835 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:38.589972 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 12:08:38.590054 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260811 12:08:38.599628 13183 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:08:38.597288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53170 (local address 127.9.221.188:36605)
0811 12:08:38.597567 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:38.597576 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:38.598151 (+ 575us) server_negotiation.cc:407] Connection header received
0811 12:08:38.598383 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:38.598389 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:38.598475 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:38.598585 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:38.599392 (+ 807us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53170: BlockingRecv error: recv got EOF from 127.0.0.1:53170 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 12:08:38.640288 13190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:38.635176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57430 (local address 127.9.221.189:42947)
0811 12:08:38.635358 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:38.635363 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:38.635861 (+ 498us) server_negotiation.cc:407] Connection header received
0811 12:08:38.636032 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:38.636036 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:38.636109 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:38.636211 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:08:38.636596 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:38.637360 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:38.638616 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:38.639041 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:38.639135 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:38.639501 (+ 366us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:08:38.639605 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:08:38.639748 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:38.639896 (+ 148us) server_negotiation.cc:299] Negotiation successful
0811 12:08:38.639958 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260811 12:08:38.648020 13183 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:08:38.646058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53174 (local address 127.9.221.188:36605)
0811 12:08:38.646242 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:38.646247 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:38.646612 (+ 365us) server_negotiation.cc:407] Connection header received
0811 12:08:38.646761 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:38.646764 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:38.646856 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:38.646937 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:38.647767 (+ 830us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53174: BlockingRecv error: recv got EOF from 127.0.0.1:53174 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 12:08:38.721938 13190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:38.715546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57446 (local address 127.9.221.189:42947)
0811 12:08:38.715831 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:38.715839 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:38.716155 (+ 316us) server_negotiation.cc:407] Connection header received
0811 12:08:38.716420 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:38.716424 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:38.716549 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:38.716653 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:08:38.717159 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:38.718121 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:38.720353 (+ 2232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:38.720754 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:38.720854 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:38.721337 (+ 483us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:08:38.721461 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:08:38.721531 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:38.721654 (+ 123us) server_negotiation.cc:299] Negotiation successful
0811 12:08:38.721717 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
W20260811 12:08:38.729925 13183 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:08:38.728075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53176 (local address 127.9.221.188:36605)
0811 12:08:38.728319 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:38.728325 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:38.728541 (+ 216us) server_negotiation.cc:407] Connection header received
0811 12:08:38.728729 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:38.728733 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:38.728805 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:38.728900 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:38.729673 (+ 773us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53176: BlockingRecv error: recv got EOF from 127.0.0.1:53176 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 12:08:38.836061 13259 catalog_manager.cc:1259] Loaded cluster ID: 9e15ed8a7a454f40837501f19978cb47
I20260811 12:08:38.836210 13259 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 72ca830006224c42bc1e148faee79368: loading cluster ID for follower catalog manager: success
I20260811 12:08:38.838663 13259 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 72ca830006224c42bc1e148faee79368: acquiring CA information for follower catalog manager: success
I20260811 12:08:38.840382 13259 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 72ca830006224c42bc1e148faee79368: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 12:08:38.872787 13190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:38.866384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57452 (local address 127.9.221.189:42947)
0811 12:08:38.866709 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:38.866718 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:38.867858 (+ 1140us) server_negotiation.cc:407] Connection header received
0811 12:08:38.868111 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:38.868115 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:38.868185 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:38.868288 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:08:38.868754 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:38.869497 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:38.870911 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:38.871424 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:38.871562 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:38.872006 (+ 444us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:08:38.872107 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:08:38.872207 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:38.872448 (+ 241us) server_negotiation.cc:299] Negotiation successful
0811 12:08:38.872528 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 12:08:38.885792 13183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:38.879787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53182 (local address 127.9.221.188:36605)
0811 12:08:38.880013 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:38.880020 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:38.880600 (+ 580us) server_negotiation.cc:407] Connection header received
0811 12:08:38.880795 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:38.880799 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:38.881106 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:38.881212 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:08:38.881641 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:38.882337 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:38.883677 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:38.884063 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:38.884154 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:38.884634 (+ 480us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:08:38.885160 (+ 526us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:08:38.885268 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:38.885397 (+ 129us) server_negotiation.cc:299] Negotiation successful
0811 12:08:38.885462 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 12:08:38.900745 13190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:38.893761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57460 (local address 127.9.221.189:42947)
0811 12:08:38.893956 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:38.893962 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:38.894649 (+ 687us) server_negotiation.cc:407] Connection header received
0811 12:08:38.894865 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:38.894872 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:38.894988 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:38.895095 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:08:38.895788 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:38.896806 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:38.898662 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:38.899307 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:38.899426 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:38.899947 (+ 521us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:08:38.900098 (+ 151us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:08:38.900190 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:38.900342 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 12:08:38.900429 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 12:08:38.901115 13466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:38.892456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56854 (local address 127.9.221.190:45399)
0811 12:08:38.893208 (+ 752us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:38.893217 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:38.893878 (+ 661us) server_negotiation.cc:407] Connection header received
0811 12:08:38.894014 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:38.894032 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:38.894269 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:38.894482 (+ 213us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:08:38.895133 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:38.896127 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:38.897964 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:38.898712 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:38.898861 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:38.899481 (+ 620us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:08:38.899977 (+ 496us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:08:38.900094 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:38.900629 (+ 535us) server_negotiation.cc:299] Negotiation successful
0811 12:08:38.900731 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":419,"threads_started":1}
I20260811 12:08:38.902348 13183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:38.892530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53188 (local address 127.9.221.188:36605)
0811 12:08:38.892739 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:38.892748 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:38.893828 (+ 1080us) server_negotiation.cc:407] Connection header received
0811 12:08:38.894914 (+ 1086us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:38.894919 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:38.894994 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:38.895089 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:08:38.896794 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:38.897940 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:38.900044 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:38.900685 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:38.900817 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:38.901521 (+ 704us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:08:38.901645 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:08:38.901750 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:38.901960 (+ 210us) server_negotiation.cc:299] Negotiation successful
0811 12:08:38.902057 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 12:08:38.906863 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 12804
I20260811 12:08:38.908249 12910 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:08:39.019618 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 12804
I20260811 12:08:39.051810 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 12934
I20260811 12:08:39.052995 13036 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:08:39.161921 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 12934
I20260811 12:08:39.192559 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 13057
I20260811 12:08:39.193771 13171 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:08:39.321390 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 13057
2026-08-11T12:08:39Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4955 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:08:39 dist-test-slave-mjjr krb5kdc[13475](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:08:39 dist-test-slave-mjjr krb5kdc[13475](info): set up 2 sockets
Aug 11 12:08:39 dist-test-slave-mjjr krb5kdc[13475](info): commencing operation
krb5kdc: starting...
W20260811 12:08:40.677681 10102 mini_kdc.cc:121] Time spent starting KDC: real 1.304s user 0.001s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/krb5kdc/test-user.keytab.
Aug 11 12:08:40 dist-test-slave-mjjr krb5kdc[13475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450120, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:08:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:08:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.221.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
Entry for principal kudu/127.9.221.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab.
I20260811 12:08:41.004999 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:40895
--webserver_interface=127.9.221.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:39097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:40895,127.9.221.189:44483,127.9.221.188:35005
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:08:41.348086 13495 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 12:08:41.348445 13495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:08:41.348536 13495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:08:41.348637 13495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:08:41.356653 13495 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:08:41.356783 13495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:08:41.356850 13495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:08:41.356911 13495 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:08:41.356984 13495 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:08:41.366865 13495 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:39097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:40895,127.9.221.189:44483,127.9.221.188:35005
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.190.keytab
--principal=kudu/127.9.221.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.190:40895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.221.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:08:41.368130 13495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:08:41.369443 13495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:08:41.381642 13508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:41.381686 13505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:41.381577 13506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:08:41.384905 13495 server_base.cc:1034] running on GCE node
I20260811 12:08:41.385574 13495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:08:41.386842 13495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:08:41.388060 13495 hybrid_clock.cc:648] HybridClock initialized: now 1786450121388004 us; error 31 us; skew 500 ppm
I20260811 12:08:41.388383 13495 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:08:41 dist-test-slave-mjjr krb5kdc[13475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450121, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:08:41.391526 13495 init.cc:377] Logged in from keytab as kudu/127.9.221.190@KRBTEST.COM (short username kudu)
I20260811 12:08:41.393090 13495 webserver.cc:460] Webserver started at http://127.9.221.190:45217/ using document root <none> and password file <none>
I20260811 12:08:41.393788 13495 fs_manager.cc:359] Metadata directory not provided
I20260811 12:08:41.393922 13495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:08:41.394251 13495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:08:41.397213 13495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-0/data/instance:
uuid: "ba6ae421929648f69827b8bd3621fbdc"
format_stamp: "Formatted at 2026-08-11 12:08:41 on dist-test-slave-mjjr"
server_key: "8e66bbd5a773719d61d73f30897d7d07"
server_key_iv: "40bb2f3be7bf57ea9d9457144b4fc0dc"
server_key_version: "encryptionkey@0"
I20260811 12:08:41.398000 13495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-0/wal/instance:
uuid: "ba6ae421929648f69827b8bd3621fbdc"
format_stamp: "Formatted at 2026-08-11 12:08:41 on dist-test-slave-mjjr"
server_key: "8e66bbd5a773719d61d73f30897d7d07"
server_key_iv: "40bb2f3be7bf57ea9d9457144b4fc0dc"
server_key_version: "encryptionkey@0"
I20260811 12:08:41.403026 13495 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 12:08:41.406517 13523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:08:41.407816 13495 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 12:08:41.407999 13495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-0/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-0/wal
uuid: "ba6ae421929648f69827b8bd3621fbdc"
format_stamp: "Formatted at 2026-08-11 12:08:41 on dist-test-slave-mjjr"
server_key: "8e66bbd5a773719d61d73f30897d7d07"
server_key_iv: "40bb2f3be7bf57ea9d9457144b4fc0dc"
server_key_version: "encryptionkey@0"
I20260811 12:08:41.408211 13495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:08:41.420986 13495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:08:41.429486 13495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:08:41.429771 13495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:08:41.448746 13495 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.190:40895
I20260811 12:08:41.448776 13612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.190:40895 every 8 connection(s)
I20260811 12:08:41.450037 13495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-0/data/info.pb
I20260811 12:08:41.453560 13614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:08:41.457304 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 13495
I20260811 12:08:41.457506 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-0/wal/instance
I20260811 12:08:41.458070 10102 external_mini_cluster.cc:1428] Setting key a44c91ff8d595bb74bfd151aa357572d
I20260811 12:08:41.461592 13614 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40895 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:08:41 dist-test-slave-mjjr krb5kdc[13475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450121, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:08:41.485018 13618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:41.465297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:40895 (local address 127.0.0.1:45416)
0811 12:08:41.466144 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:41.466194 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 12:08:41.468254 (+ 2060us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:08:41.469022 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:08:41.469028 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:08:41.469724 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:08:41.470054 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:08:41.470069 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:08:41.471589 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:08:41.471597 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:08:41.472983 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:08:41.473000 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:08:41.473913 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:08:41.473920 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:08:41.474117 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:41.474798 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:08:41.474838 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:08:41.477553 (+ 2715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:08:41.480831 (+ 3278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:08:41.480844 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:08:41.480855 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:08:41.481264 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:08:41.481679 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:08:41.481685 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:08:41.481688 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:08:41.481842 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:08:41.482581 (+ 739us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:08:41.482588 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:08:41.482823 (+ 235us) client_negotiation.cc:769] Sending connection context
0811 12:08:41.483065 (+ 242us) client_negotiation.cc:240] Negotiation successful
0811 12:08:41.483711 (+ 646us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"mutex_wait_us":126,"thread_start_us":359,"threads_started":1}
I20260811 12:08:41.485049 13619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:41.465298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45416 (local address 127.9.221.190:40895)
0811 12:08:41.466938 (+ 1640us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:41.466947 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:41.466996 (+ 49us) server_negotiation.cc:407] Connection header received
0811 12:08:41.468606 (+ 1610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:41.468612 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:41.468815 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:41.469011 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:41.470264 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:41.471437 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:41.473128 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:41.473657 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:41.473884 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:41.477786 (+ 3902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:41.477795 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:41.477806 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:41.477847 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:41.480678 (+ 2831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:41.481416 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:41.481429 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:41.481431 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:41.481570 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:41.481955 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:41.481960 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:41.481965 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:41.482434 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:41.482687 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:41.483200 (+ 513us) server_negotiation.cc:299] Negotiation successful
0811 12:08:41.483513 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1417,"thread_start_us":1199,"threads_started":1}
I20260811 12:08:41.488256 13614 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.221.189" port: 44483 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:08:41.489493 13618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:41.489094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:44483 (local address 127.0.0.1:54240)
0811 12:08:41.489280 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:41.489384 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.189:44483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260811 12:08:41.489768 13525 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.221.189:44483: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.9.221.189@KRBTEST.COM; defaulting to no policy
I20260811 12:08:41.491379 13618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:41.490974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:44483 (local address 127.0.0.1:54242)
0811 12:08:41.491163 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:41.491248 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.189:44483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 12:08:41.491582 13614 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.221.189:44483: Network error: Client connection negotiation failed: client connection to 127.9.221.189:44483: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.189.keytab.
Entry for principal kudu/127.9.221.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.189.keytab.
I20260811 12:08:41.530386 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.189:44483
--webserver_interface=127.9.221.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.189.keytab
--principal=kudu/127.9.221.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:39097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:40895,127.9.221.189:44483,127.9.221.188:35005
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:08:41.540983 13614 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.221.189" port: 44483 } attempt: 1
I20260811 12:08:41.542514 13618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:41.541856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:44483 (local address 127.0.0.1:54254)
0811 12:08:41.542159 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:41.542306 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.189:44483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260811 12:08:41.543577 13618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:41.543178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:44483 (local address 127.0.0.1:54266)
0811 12:08:41.543370 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:41.543457 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.189:44483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 12:08:41.543804 13614 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.221.189:44483: Network error: Client connection negotiation failed: client connection to 127.9.221.189:44483: connect: Connection refused (error 111)
I20260811 12:08:41.612103 13614 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.221.189" port: 44483 } attempt: 2
I20260811 12:08:41.613449 13618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:41.612852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:44483 (local address 127.0.0.1:54278)
0811 12:08:41.613124 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:41.613265 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.189:44483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260811 12:08:41.614511 13618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:41.614104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:44483 (local address 127.0.0.1:54284)
0811 12:08:41.614269 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:41.614367 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.189:44483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260811 12:08:41.614737 13614 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.221.189:44483: Network error: Client connection negotiation failed: client connection to 127.9.221.189:44483: connect: Connection refused (error 111)
I20260811 12:08:41.706063 13614 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.221.189" port: 44483 } attempt: 3
I20260811 12:08:41.707401 13618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:41.706857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:44483 (local address 127.0.0.1:54286)
0811 12:08:41.707100 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:41.707218 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.189:44483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260811 12:08:41.708577 13618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:41.708163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:44483 (local address 127.0.0.1:54294)
0811 12:08:41.708334 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:41.708418 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.189:44483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 12:08:41.708801 13614 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.221.189:44483: Network error: Client connection negotiation failed: client connection to 127.9.221.189:44483: connect: Connection refused (error 111)
I20260811 12:08:41.852074 13614 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.221.189" port: 44483 } attempt: 4
I20260811 12:08:41.853516 13618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:41.852821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:44483 (local address 127.0.0.1:54300)
0811 12:08:41.853201 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:41.853332 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.189:44483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194}
I20260811 12:08:41.854621 13618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:41.854175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:44483 (local address 127.0.0.1:54304)
0811 12:08:41.854354 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:41.854463 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.189:44483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260811 12:08:41.854825 13614 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.221.189:44483: Network error: Client connection negotiation failed: client connection to 127.9.221.189:44483: connect: Connection refused (error 111)
W20260811 12:08:41.880267 13625 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 12:08:41.880568 13625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:08:41.880648 13625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:08:41.880743 13625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:08:41.888054 13625 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:08:41.888144 13625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:08:41.888208 13625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:08:41.888265 13625 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:08:41.888332 13625 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:08:41.898591 13625 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:39097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:40895,127.9.221.189:44483,127.9.221.188:35005
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.189.keytab
--principal=kudu/127.9.221.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.189:44483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.221.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:08:41.899983 13625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:08:41.901362 13625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:08:41.913344 13633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:41.913300 13635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:41.913281 13638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:08:41.914252 13625 server_base.cc:1034] running on GCE node
I20260811 12:08:41.914988 13625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:08:41.916344 13625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:08:41.917565 13625 hybrid_clock.cc:648] HybridClock initialized: now 1786450121917506 us; error 23 us; skew 500 ppm
I20260811 12:08:41.917834 13625 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:08:41 dist-test-slave-mjjr krb5kdc[13475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450121, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:08:41.920929 13625 init.cc:377] Logged in from keytab as kudu/127.9.221.189@KRBTEST.COM (short username kudu)
I20260811 12:08:41.922377 13625 webserver.cc:460] Webserver started at http://127.9.221.189:43295/ using document root <none> and password file <none>
I20260811 12:08:41.923071 13625 fs_manager.cc:359] Metadata directory not provided
I20260811 12:08:41.923223 13625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:08:41.923542 13625 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:08:41.926530 13625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-1/data/instance:
uuid: "ad76e749a4b242ebbfbfe5cabeec19ad"
format_stamp: "Formatted at 2026-08-11 12:08:41 on dist-test-slave-mjjr"
server_key: "c7ebbc93332eb91d328177d2bb2058c0"
server_key_iv: "811445cbcfc7b4d37d106cdf8f977f2d"
server_key_version: "encryptionkey@0"
I20260811 12:08:41.927325 13625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-1/wal/instance:
uuid: "ad76e749a4b242ebbfbfe5cabeec19ad"
format_stamp: "Formatted at 2026-08-11 12:08:41 on dist-test-slave-mjjr"
server_key: "c7ebbc93332eb91d328177d2bb2058c0"
server_key_iv: "811445cbcfc7b4d37d106cdf8f977f2d"
server_key_version: "encryptionkey@0"
I20260811 12:08:41.932401 13625 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 12:08:41.935798 13650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:08:41.937184 13625 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 12:08:41.937386 13625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-1/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-1/wal
uuid: "ad76e749a4b242ebbfbfe5cabeec19ad"
format_stamp: "Formatted at 2026-08-11 12:08:41 on dist-test-slave-mjjr"
server_key: "c7ebbc93332eb91d328177d2bb2058c0"
server_key_iv: "811445cbcfc7b4d37d106cdf8f977f2d"
server_key_version: "encryptionkey@0"
I20260811 12:08:41.937603 13625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:08:41.955482 13625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:08:41.959450 13625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:08:41.959842 13625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:08:41.978257 13625 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.189:44483
I20260811 12:08:41.978277 13736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.189:44483 every 8 connection(s)
I20260811 12:08:41.979660 13625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-1/data/info.pb
I20260811 12:08:41.982018 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 13625
I20260811 12:08:41.982239 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-1/wal/instance
I20260811 12:08:41.982771 10102 external_mini_cluster.cc:1428] Setting key edc196b91904933718ab5df8910a72ea
I20260811 12:08:41.983577 13739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:08:41.989938 13739 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40895 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:08:42 dist-test-slave-mjjr krb5kdc[13475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450121, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.189@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:08:42.013294 13746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:41.992469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45418 (local address 127.9.221.190:40895)
0811 12:08:41.993008 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:41.993014 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:41.993989 (+ 975us) server_negotiation.cc:407] Connection header received
0811 12:08:41.996731 (+ 2742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:41.996735 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:41.996799 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:41.996893 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:41.998467 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:41.999744 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:42.001235 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:42.001988 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:42.002159 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:42.005561 (+ 3402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:42.005589 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:42.005596 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:42.005653 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:42.009642 (+ 3989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:42.010363 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:42.010372 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:42.010388 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:42.010539 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:42.010973 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:42.010980 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:42.010986 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:42.011416 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:42.011581 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:42.012518 (+ 937us) server_negotiation.cc:299] Negotiation successful
0811 12:08:42.012789 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":277,"threads_started":1}
I20260811 12:08:42.014043 13748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:41.992966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:40895 (local address 127.0.0.1:45418)
0811 12:08:41.993766 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:41.993824 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 12:08:41.996464 (+ 2640us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:08:41.997166 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:08:41.997177 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:08:41.997694 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:08:41.998224 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:08:41.998261 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:08:41.999929 (+ 1668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:08:41.999933 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:08:42.001082 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:08:42.001094 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:08:42.002195 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:08:42.002200 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:08:42.002384 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:42.003087 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:08:42.003109 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:08:42.005331 (+ 2222us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:08:42.009786 (+ 4455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:08:42.009792 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:08:42.009803 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:08:42.010228 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:08:42.010658 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:08:42.010661 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:08:42.010663 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:08:42.010862 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:08:42.011620 (+ 758us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:08:42.011626 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:08:42.012096 (+ 470us) client_negotiation.cc:769] Sending connection context
0811 12:08:42.012381 (+ 285us) client_negotiation.cc:240] Negotiation successful
0811 12:08:42.012804 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":300,"threads_started":1}
I20260811 12:08:42.015838 13739 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.221.189" port: 44483 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.9.221.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.221.188@KRBTEST.COM" created.
Aug 11 12:08:42 dist-test-slave-mjjr krb5kdc[13475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450121, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.189@KRBTEST.COM for kudu/127.9.221.189@KRBTEST.COM
I20260811 12:08:42.030085 13748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:42.016746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:44483 (local address 127.0.0.1:54312)
0811 12:08:42.016928 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:42.016952 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 12:08:42.017075 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:08:42.017846 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:08:42.017850 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:08:42.018319 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:08:42.018453 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:08:42.018460 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:08:42.020014 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:08:42.020020 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:08:42.020866 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:08:42.020877 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:08:42.021699 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:08:42.021705 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:08:42.021891 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:42.022466 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:08:42.022483 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:08:42.024502 (+ 2019us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:08:42.027686 (+ 3184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:08:42.027693 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:08:42.027697 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:08:42.028053 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:08:42.028435 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:08:42.028439 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:08:42.028443 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:08:42.028576 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:08:42.029151 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:08:42.029160 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:08:42.029288 (+ 128us) client_negotiation.cc:769] Sending connection context
0811 12:08:42.029440 (+ 152us) client_negotiation.cc:240] Negotiation successful
0811 12:08:42.029705 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 12:08:42.030241 13753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:42.016835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54312 (local address 127.9.221.189:44483)
0811 12:08:42.017463 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:42.017471 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:42.017505 (+ 34us) server_negotiation.cc:407] Connection header received
0811 12:08:42.017545 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:42.017551 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:42.017705 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:42.017870 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:42.018626 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:42.019883 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:42.021013 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:42.021530 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:42.021674 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:42.024717 (+ 3043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:42.024727 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:42.024740 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:42.024766 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:42.027547 (+ 2781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:42.028184 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:42.028199 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:42.028201 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:42.028318 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:42.028686 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:42.028692 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:42.028696 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:42.029027 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:42.029240 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:42.029578 (+ 338us) server_negotiation.cc:299] Negotiation successful
0811 12:08:42.029829 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":438,"thread_start_us":270,"threads_started":1}
I20260811 12:08:42.032639 13739 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.221.188" port: 35005 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:08:42.033972 13748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:42.033411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:35005 (local address 127.0.0.1:40170)
0811 12:08:42.033611 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:42.033760 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:35005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 12:08:42.034267 13658 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.221.188:35005: connect: Connection refused (error 111)
I20260811 12:08:42.035728 13748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:42.035373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:35005 (local address 127.0.0.1:40176)
0811 12:08:42.035507 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:42.035579 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:35005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 12:08:42.035989 13739 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.221.188:35005: Network error: Client connection negotiation failed: client connection to 127.9.221.188:35005: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.221.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.188.keytab.
Entry for principal kudu/127.9.221.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.188.keytab.
I20260811 12:08:42.058261 10102 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
/tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.188:35005
--webserver_interface=127.9.221.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.188.keytab
--principal=kudu/127.9.221.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.221.148:39097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.221.190:40895,127.9.221.189:44483,127.9.221.188:35005
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:08:42.085278 13739 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.221.188" port: 35005 } attempt: 1
I20260811 12:08:42.086630 13748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:42.085964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:35005 (local address 127.0.0.1:40178)
0811 12:08:42.086229 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:42.086391 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:35005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260811 12:08:42.087819 13748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:42.087323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:35005 (local address 127.0.0.1:40192)
0811 12:08:42.087534 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:42.087663 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:35005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260811 12:08:42.088061 13739 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.221.188:35005: Network error: Client connection negotiation failed: client connection to 127.9.221.188:35005: connect: Connection refused (error 111)
I20260811 12:08:42.154102 13614 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.221.189" port: 44483 } attempt: 5
I20260811 12:08:42.156281 13739 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.221.188" port: 35005 } attempt: 2
I20260811 12:08:42.157301 13748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:42.156815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:35005 (local address 127.0.0.1:40200)
0811 12:08:42.157013 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:42.157115 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:35005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 12:08:42.158350 13748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:42.158030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:35005 (local address 127.0.0.1:40206)
0811 12:08:42.158167 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:42.158242 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:35005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 12:08:42.158630 13739 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.221.188:35005: Network error: Client connection negotiation failed: client connection to 127.9.221.188:35005: connect: Connection refused (error 111)
Aug 11 12:08:42 dist-test-slave-mjjr krb5kdc[13475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450121, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for kudu/127.9.221.189@KRBTEST.COM
I20260811 12:08:42.169744 13618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:42.154877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:44483 (local address 127.0.0.1:54324)
0811 12:08:42.155124 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:42.155149 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 12:08:42.155289 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:08:42.155744 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:08:42.155747 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:08:42.156121 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:08:42.156418 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:08:42.156433 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:08:42.157789 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:08:42.157795 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:08:42.159215 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:08:42.159233 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:08:42.160026 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:08:42.160033 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:08:42.160247 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:42.160896 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:08:42.160926 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:08:42.163332 (+ 2406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:08:42.167137 (+ 3805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:08:42.167141 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:08:42.167144 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:08:42.167529 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:08:42.167987 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:08:42.167990 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:08:42.167993 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:08:42.168129 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:08:42.168915 (+ 786us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:08:42.168919 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:08:42.169077 (+ 158us) client_negotiation.cc:769] Sending connection context
0811 12:08:42.169246 (+ 169us) client_negotiation.cc:240] Negotiation successful
0811 12:08:42.169468 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87}
I20260811 12:08:42.169909 13753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:42.155120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54324 (local address 127.9.221.189:44483)
0811 12:08:42.155354 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:42.155360 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:42.155392 (+ 32us) server_negotiation.cc:407] Connection header received
0811 12:08:42.155459 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:42.155464 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:42.155544 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:42.155681 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:42.156613 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:42.157623 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:42.159373 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:42.159877 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:42.159977 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:42.163532 (+ 3555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:42.163539 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:42.163544 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:42.163582 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:42.166990 (+ 3408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:42.167684 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:42.167693 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:42.167699 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:42.167868 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:42.168250 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:42.168256 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:42.168261 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:42.168720 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:42.168883 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:42.169397 (+ 514us) server_negotiation.cc:299] Negotiation successful
0811 12:08:42.169630 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 12:08:42.170931 13614 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.221.188" port: 35005 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:08:42.172008 13618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:42.171585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:35005 (local address 127.0.0.1:40216)
0811 12:08:42.171785 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:42.171879 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:35005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260811 12:08:42.172977 13618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:42.172661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:35005 (local address 127.0.0.1:40230)
0811 12:08:42.172788 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:42.172882 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:35005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 12:08:42.173202 13614 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.221.188:35005: Network error: Client connection negotiation failed: client connection to 127.9.221.188:35005: connect: Connection refused (error 111)
I20260811 12:08:42.224434 13614 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.221.188" port: 35005 } attempt: 1
I20260811 12:08:42.225613 13618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:42.225143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:35005 (local address 127.0.0.1:40246)
0811 12:08:42.225350 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:42.225445 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:35005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 12:08:42.226744 13618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:42.226268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:35005 (local address 127.0.0.1:40256)
0811 12:08:42.226486 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:42.226591 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:35005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260811 12:08:42.226980 13614 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.221.188:35005: Network error: Client connection negotiation failed: client connection to 127.9.221.188:35005: connect: Connection refused (error 111)
I20260811 12:08:42.249850 13739 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.221.188" port: 35005 } attempt: 3
I20260811 12:08:42.250896 13748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:42.250432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:35005 (local address 127.0.0.1:40270)
0811 12:08:42.250643 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:42.250740 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:35005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 12:08:42.251943 13748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:42.251516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:35005 (local address 127.0.0.1:40272)
0811 12:08:42.251761 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:42.251832 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:35005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148}
W20260811 12:08:42.252185 13739 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.221.188:35005: Network error: Client connection negotiation failed: client connection to 127.9.221.188:35005: connect: Connection refused (error 111)
I20260811 12:08:42.295255 13614 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.221.188" port: 35005 } attempt: 2
I20260811 12:08:42.296705 13618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:42.296055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:35005 (local address 127.0.0.1:40278)
0811 12:08:42.296334 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:42.296491 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:35005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260811 12:08:42.297837 13618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:42.297370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:35005 (local address 127.0.0.1:40290)
0811 12:08:42.297583 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:42.297678 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:35005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
W20260811 12:08:42.298063 13614 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.221.188:35005: Network error: Client connection negotiation failed: client connection to 127.9.221.188:35005: connect: Connection refused (error 111)
I20260811 12:08:42.395396 13739 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.221.188" port: 35005 } attempt: 4
I20260811 12:08:42.396564 13748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:42.396087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:35005 (local address 127.0.0.1:40302)
0811 12:08:42.396327 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:42.396415 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:35005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260811 12:08:42.397842 13748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:42.397378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:35005 (local address 127.0.0.1:40308)
0811 12:08:42.397586 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:42.397682 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:35005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 12:08:42.398176 13739 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.221.188:35005: Network error: Client connection negotiation failed: client connection to 127.9.221.188:35005: connect: Connection refused (error 111)
I20260811 12:08:42.404275 13614 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.221.188" port: 35005 } attempt: 3
W20260811 12:08:42.404922 13757 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 12:08:42.405370 13757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
I20260811 12:08:42.405411 13618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:42.404824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:35005 (local address 127.0.0.1:40314)
0811 12:08:42.405072 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:42.405183 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:35005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
W20260811 12:08:42.405489 13757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:08:42.405611 13757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 12:08:42.406369 13618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:42.406018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:35005 (local address 127.0.0.1:40324)
0811 12:08:42.406203 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:42.406273 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.221.188:35005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260811 12:08:42.406610 13614 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.221.188:35005: Network error: Client connection negotiation failed: client connection to 127.9.221.188:35005: connect: Connection refused (error 111)
W20260811 12:08:42.413210 13757 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:08:42.413285 13757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:08:42.413415 13757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:08:42.413503 13757 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:08:42.413607 13757 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:08:42.423241 13757 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.221.148:39097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.221.190:40895,127.9.221.189:44483,127.9.221.188:35005
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/krb5kdc/kudu_127.9.221.188.keytab
--principal=kudu/127.9.221.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.221.188:35005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.221.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:08:42.424464 13757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:08:42.425776 13757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:08:42.437561 13772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:42.437630 13767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:42.437664 13768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:08:42.438431 13757 server_base.cc:1034] running on GCE node
I20260811 12:08:42.439226 13757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:08:42.440574 13757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:08:42.441823 13757 hybrid_clock.cc:648] HybridClock initialized: now 1786450122441741 us; error 59 us; skew 500 ppm
I20260811 12:08:42.442170 13757 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:08:42 dist-test-slave-mjjr krb5kdc[13475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450122, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:08:42.445381 13757 init.cc:377] Logged in from keytab as kudu/127.9.221.188@KRBTEST.COM (short username kudu)
I20260811 12:08:42.446775 13757 webserver.cc:460] Webserver started at http://127.9.221.188:42137/ using document root <none> and password file <none>
I20260811 12:08:42.447434 13757 fs_manager.cc:359] Metadata directory not provided
I20260811 12:08:42.447566 13757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:08:42.447924 13757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:08:42.450841 13757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-2/data/instance:
uuid: "8de16c51296b48e29ec58c2f44e32c23"
format_stamp: "Formatted at 2026-08-11 12:08:42 on dist-test-slave-mjjr"
server_key: "36175d7edae929bc70afbc2383dfa355"
server_key_iv: "675dbea12189b705b0806de6ebef5aa9"
server_key_version: "encryptionkey@0"
I20260811 12:08:42.451586 13757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-2/wal/instance:
uuid: "8de16c51296b48e29ec58c2f44e32c23"
format_stamp: "Formatted at 2026-08-11 12:08:42 on dist-test-slave-mjjr"
server_key: "36175d7edae929bc70afbc2383dfa355"
server_key_iv: "675dbea12189b705b0806de6ebef5aa9"
server_key_version: "encryptionkey@0"
I20260811 12:08:42.456434 13757 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 12:08:42.459673 13782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:08:42.460911 13757 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 12:08:42.461113 13757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-2/data,/tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-2/wal
uuid: "8de16c51296b48e29ec58c2f44e32c23"
format_stamp: "Formatted at 2026-08-11 12:08:42 on dist-test-slave-mjjr"
server_key: "36175d7edae929bc70afbc2383dfa355"
server_key_iv: "675dbea12189b705b0806de6ebef5aa9"
server_key_version: "encryptionkey@0"
I20260811 12:08:42.461325 13757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:08:42.503408 13757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:08:42.507443 13757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:08:42.507741 13757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:08:42.526448 13757 rpc_server.cc:307] RPC server started. Bound to: 127.9.221.188:35005
I20260811 12:08:42.526482 13877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.221.188:35005 every 8 connection(s)
I20260811 12:08:42.527870 13757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-2/data/info.pb
I20260811 12:08:42.530936 10102 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu as pid 13757
I20260811 12:08:42.531149 10102 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGvgjmQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449743084027-10102-0/minicluster-data/master-2/wal/instance
I20260811 12:08:42.531790 10102 external_mini_cluster.cc:1428] Setting key 1c3d7754f0c303965a859609a9f5897f
I20260811 12:08:42.531840 13883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:08:42.541304 13883 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40895 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:08:42 dist-test-slave-mjjr krb5kdc[13475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450120, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.188@KRBTEST.COM
Aug 11 12:08:42 dist-test-slave-mjjr krb5kdc[13475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450122, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.188@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:08:42.555035 13887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:42.533573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40336 (local address 127.9.221.188:35005)
0811 12:08:42.534379 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:42.534392 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:42.534445 (+ 53us) server_negotiation.cc:407] Connection header received
0811 12:08:42.536470 (+ 2025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:42.536489 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:42.536849 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:42.537161 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:42.538910 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:42.540233 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:42.542104 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:42.542842 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:42.543013 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:42.547762 (+ 4749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:42.547773 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:42.547782 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:42.547821 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:42.550962 (+ 3141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:42.551808 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:42.551815 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:42.551821 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:42.552024 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:42.552484 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:42.552487 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:42.552489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:42.552945 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:42.553258 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:42.553702 (+ 444us) server_negotiation.cc:299] Negotiation successful
0811 12:08:42.554093 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":400,"threads_started":1}
I20260811 12:08:42.558667 13889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:42.544027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.190:40895 (local address 127.0.0.1:45434)
0811 12:08:42.544706 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:42.544746 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 12:08:42.544903 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:08:42.545616 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:08:42.545623 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:08:42.546226 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:08:42.546483 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:08:42.546492 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:08:42.547544 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:08:42.547547 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:08:42.548657 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:08:42.548669 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:08:42.549417 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:08:42.549423 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:08:42.549617 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:42.550196 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:08:42.550213 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:08:42.552715 (+ 2502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:08:42.556040 (+ 3325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:08:42.556054 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:08:42.556060 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:08:42.556470 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:08:42.556871 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:08:42.556876 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:08:42.556880 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:08:42.557020 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:08:42.557667 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:08:42.557674 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:08:42.557898 (+ 224us) client_negotiation.cc:769] Sending connection context
0811 12:08:42.558106 (+ 208us) client_negotiation.cc:240] Negotiation successful
0811 12:08:42.558404 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":327,"threads_started":1}
I20260811 12:08:42.559434 13891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:42.544105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45434 (local address 127.9.221.190:40895)
0811 12:08:42.545284 (+ 1179us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:42.545289 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:42.545304 (+ 15us) server_negotiation.cc:407] Connection header received
0811 12:08:42.545352 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:42.545355 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:42.545431 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:42.545552 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:42.546633 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:42.547367 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:42.548842 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:42.549236 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:42.549377 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:42.552907 (+ 3530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:42.552913 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:42.552916 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:42.552948 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:42.555915 (+ 2967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:42.556612 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:42.556622 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:42.556625 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:42.556751 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:42.557143 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:42.557149 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:42.557152 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:42.557518 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:42.557693 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:42.558982 (+ 1289us) server_negotiation.cc:299] Negotiation successful
0811 12:08:42.559153 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1080,"thread_start_us":962,"threads_started":1}
I20260811 12:08:42.560748 13883 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.221.189" port: 44483 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:08:42 dist-test-slave-mjjr krb5kdc[13475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450122, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.188@KRBTEST.COM for kudu/127.9.221.189@KRBTEST.COM
Aug 11 12:08:42 dist-test-slave-mjjr krb5kdc[13475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450120, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.190@KRBTEST.COM
I20260811 12:08:42.574381 13889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:42.561520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.189:44483 (local address 127.0.0.1:54336)
0811 12:08:42.561698 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:42.561717 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:08:42.561824 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:08:42.562137 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:08:42.562142 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:08:42.562593 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:08:42.562768 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:08:42.562778 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:08:42.563724 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:08:42.563728 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:08:42.564643 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:08:42.564654 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:08:42.565388 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:08:42.565394 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:08:42.565601 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:42.566131 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:08:42.566147 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:08:42.568345 (+ 2198us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:08:42.571318 (+ 2973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:08:42.571324 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:08:42.571328 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:08:42.571752 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:08:42.572692 (+ 940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:08:42.572695 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:08:42.572698 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:08:42.572877 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:08:42.573426 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:08:42.573430 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:08:42.573585 (+ 155us) client_negotiation.cc:769] Sending connection context
0811 12:08:42.573722 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 12:08:42.574011 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 12:08:42.574429 13753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:42.561532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54336 (local address 127.9.221.189:44483)
0811 12:08:42.561723 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:42.561727 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:42.561769 (+ 42us) server_negotiation.cc:407] Connection header received
0811 12:08:42.561946 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:42.561950 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:42.562009 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:42.562082 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:42.562926 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:42.563556 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:42.564786 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:42.565203 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:42.565386 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:42.568522 (+ 3136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:42.568527 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:42.568530 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:42.568561 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:42.571186 (+ 2625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:42.571874 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:42.571880 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:42.571884 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:42.572032 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:42.573006 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:42.573011 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:42.573014 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:42.573288 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:42.573458 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:42.573829 (+ 371us) server_negotiation.cc:299] Negotiation successful
0811 12:08:42.574040 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 12:08:42.574945 13892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:42.557870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45442 (local address 127.9.221.190:40895)
0811 12:08:42.558423 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:42.558427 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:42.558445 (+ 18us) server_negotiation.cc:407] Connection header received
0811 12:08:42.558481 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:42.558484 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:42.558533 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:42.558644 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:42.560139 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:42.561175 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:42.563756 (+ 2581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:42.564168 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:42.564302 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:42.569204 (+ 4902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:42.569213 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:42.569221 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:42.569267 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:42.571982 (+ 2715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:42.572716 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:42.572724 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:42.572726 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:42.572881 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:42.573386 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:42.573391 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:42.573393 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:42.573657 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:42.573945 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:42.574306 (+ 361us) server_negotiation.cc:299] Negotiation successful
0811 12:08:42.574543 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":259,"threads_started":1}
I20260811 12:08:42.575451 13883 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.221.188" port: 35005 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:08:42 dist-test-slave-mjjr krb5kdc[13475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450122, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.188@KRBTEST.COM for kudu/127.9.221.188@KRBTEST.COM
I20260811 12:08:42.583829 13614 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.221.188" port: 35005 } attempt: 4
Aug 11 12:08:42 dist-test-slave-mjjr krb5kdc[13475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450120, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.221.189@KRBTEST.COM
I20260811 12:08:42.592218 13889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:42.576450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:35005 (local address 127.0.0.1:40344)
0811 12:08:42.576614 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:42.576635 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:08:42.576744 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:08:42.577099 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:08:42.577105 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:08:42.577493 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:08:42.577625 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:08:42.577632 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:08:42.579125 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:08:42.579129 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:08:42.580426 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:08:42.580440 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:08:42.581380 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:08:42.581384 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:08:42.581548 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:42.582160 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:08:42.582177 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:08:42.584647 (+ 2470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:08:42.589342 (+ 4695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:08:42.589349 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:08:42.589355 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:08:42.589913 (+ 558us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:08:42.590354 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:08:42.590359 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:08:42.590365 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:08:42.590518 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:08:42.591162 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:08:42.591167 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:08:42.591331 (+ 164us) client_negotiation.cc:769] Sending connection context
0811 12:08:42.591516 (+ 185us) client_negotiation.cc:240] Negotiation successful
0811 12:08:42.591848 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 12:08:42.592983 13887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:42.576310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40344 (local address 127.9.221.188:35005)
0811 12:08:42.576498 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:42.576505 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:42.576707 (+ 202us) server_negotiation.cc:407] Connection header received
0811 12:08:42.576868 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:42.576872 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:42.576954 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:42.577054 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:42.577754 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:42.578993 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:42.580563 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:42.581207 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:42.581342 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:42.584865 (+ 3523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:42.584872 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:42.584878 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:42.584917 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:42.589188 (+ 4271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:42.590050 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:42.590056 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:42.590060 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:42.590177 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:42.590643 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:42.590649 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:42.590652 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:42.591024 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:42.591179 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:42.592519 (+ 1340us) server_negotiation.cc:299] Negotiation successful
0811 12:08:42.592784 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
Aug 11 12:08:42 dist-test-slave-mjjr krb5kdc[13475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450121, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.190@KRBTEST.COM for kudu/127.9.221.188@KRBTEST.COM
I20260811 12:08:42.594969 13753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:42.577196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54338 (local address 127.9.221.189:44483)
0811 12:08:42.577358 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:42.577364 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:42.577503 (+ 139us) server_negotiation.cc:407] Connection header received
0811 12:08:42.577673 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:42.577677 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:42.577764 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:42.577876 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:42.579454 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:42.580539 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:42.581992 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:42.582420 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:42.582554 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:42.587898 (+ 5344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:42.587907 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:42.587912 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:42.587956 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:42.591406 (+ 3450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:42.592179 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:42.592184 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:42.592189 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:42.592386 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:42.592849 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:42.592855 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:42.592858 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:42.593211 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:42.593492 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:42.594507 (+ 1015us) server_negotiation.cc:299] Negotiation successful
0811 12:08:42.594738 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 12:08:42.599790 10102 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:08:42.599937 13883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8de16c51296b48e29ec58c2f44e32c23: Bootstrap starting.
I20260811 12:08:42.601523 13618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:42.584427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:35005 (local address 127.0.0.1:40350)
0811 12:08:42.584689 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:42.584712 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:08:42.584856 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:08:42.585790 (+ 934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:08:42.585797 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:08:42.586291 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:08:42.586439 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:08:42.586448 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:08:42.588166 (+ 1718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:08:42.588174 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:08:42.589252 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:08:42.589293 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:08:42.590297 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:08:42.590304 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:08:42.590502 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:42.591110 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:08:42.591142 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:08:42.594059 (+ 2917us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:08:42.598610 (+ 4551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:08:42.598614 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:08:42.598618 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:08:42.599131 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:08:42.599603 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:08:42.599609 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:08:42.599612 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:08:42.599817 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:08:42.600488 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:08:42.600493 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:08:42.600691 (+ 198us) client_negotiation.cc:769] Sending connection context
0811 12:08:42.600849 (+ 158us) client_negotiation.cc:240] Negotiation successful
0811 12:08:42.601140 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139}
I20260811 12:08:42.601581 13893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:42.584463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40350 (local address 127.9.221.188:35005)
0811 12:08:42.585439 (+ 976us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:42.585446 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:42.585464 (+ 18us) server_negotiation.cc:407] Connection header received
0811 12:08:42.585499 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:42.585502 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:42.585549 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:42.585733 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:42.586630 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:42.588010 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:42.589406 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:42.590114 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:42.590256 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:42.594283 (+ 4027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:42.594290 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:42.594294 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:42.594329 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:42.598469 (+ 4140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:42.599267 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:42.599281 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:42.599284 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:42.599453 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:42.599968 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:42.599973 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:42.599977 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:42.600355 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:42.600515 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:42.600957 (+ 442us) server_negotiation.cc:299] Negotiation successful
0811 12:08:42.601155 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":883,"thread_start_us":692,"threads_started":1}
I20260811 12:08:42.604107 13883 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8de16c51296b48e29ec58c2f44e32c23: Neither blocks nor log segments found. Creating new log.
I20260811 12:08:42.605494 13883 log.cc:824] T 00000000000000000000000000000000 P 8de16c51296b48e29ec58c2f44e32c23: Log is configured to *not* fsync() on all Append() calls
I20260811 12:08:42.607034 13614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba6ae421929648f69827b8bd3621fbdc: Bootstrap starting.
I20260811 12:08:42.602850 13575 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "ba6ae421929648f69827b8bd3621fbdc"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45442
I20260811 12:08:42.609032 13883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8de16c51296b48e29ec58c2f44e32c23: No bootstrap required, opened a new log
I20260811 12:08:42.610230 13614 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba6ae421929648f69827b8bd3621fbdc: Neither blocks nor log segments found. Creating new log.
I20260811 12:08:42.611199 13614 log.cc:824] T 00000000000000000000000000000000 P ba6ae421929648f69827b8bd3621fbdc: Log is configured to *not* fsync() on all Append() calls
I20260811 12:08:42.614058 13614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba6ae421929648f69827b8bd3621fbdc: No bootstrap required, opened a new log
I20260811 12:08:42.618945 13614 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba6ae421929648f69827b8bd3621fbdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6ae421929648f69827b8bd3621fbdc" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40895 } } peers { permanent_uuid: "ad76e749a4b242ebbfbfe5cabeec19ad" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 44483 } } peers { permanent_uuid: "8de16c51296b48e29ec58c2f44e32c23" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 35005 } }
I20260811 12:08:42.619216 13614 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba6ae421929648f69827b8bd3621fbdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba6ae421929648f69827b8bd3621fbdc, State: Initialized, Role: FOLLOWER
I20260811 12:08:42.619625 13614 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba6ae421929648f69827b8bd3621fbdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6ae421929648f69827b8bd3621fbdc" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40895 } } peers { permanent_uuid: "ad76e749a4b242ebbfbfe5cabeec19ad" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 44483 } } peers { permanent_uuid: "8de16c51296b48e29ec58c2f44e32c23" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 35005 } }
I20260811 12:08:42.619786 13883 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8de16c51296b48e29ec58c2f44e32c23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6ae421929648f69827b8bd3621fbdc" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40895 } } peers { permanent_uuid: "ad76e749a4b242ebbfbfe5cabeec19ad" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 44483 } } peers { permanent_uuid: "8de16c51296b48e29ec58c2f44e32c23" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 35005 } }
I20260811 12:08:42.620164 13883 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8de16c51296b48e29ec58c2f44e32c23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8de16c51296b48e29ec58c2f44e32c23, State: Initialized, Role: FOLLOWER
I20260811 12:08:42.620618 13883 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8de16c51296b48e29ec58c2f44e32c23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6ae421929648f69827b8bd3621fbdc" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40895 } } peers { permanent_uuid: "ad76e749a4b242ebbfbfe5cabeec19ad" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 44483 } } peers { permanent_uuid: "8de16c51296b48e29ec58c2f44e32c23" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 35005 } }
I20260811 12:08:42.620580 13898 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba6ae421929648f69827b8bd3621fbdc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6ae421929648f69827b8bd3621fbdc" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40895 } } peers { permanent_uuid: "ad76e749a4b242ebbfbfe5cabeec19ad" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 44483 } } peers { permanent_uuid: "8de16c51296b48e29ec58c2f44e32c23" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 35005 } } }
I20260811 12:08:42.621024 13898 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba6ae421929648f69827b8bd3621fbdc [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:08:42.621680 13614 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba6ae421929648f69827b8bd3621fbdc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:08:42.621811 13575 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba6ae421929648f69827b8bd3621fbdc [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 12:08:42.622027 13575 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba6ae421929648f69827b8bd3621fbdc [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:08:42.622880 13883 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8de16c51296b48e29ec58c2f44e32c23 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:08:42.623194 13899 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8de16c51296b48e29ec58c2f44e32c23 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6ae421929648f69827b8bd3621fbdc" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40895 } } peers { permanent_uuid: "ad76e749a4b242ebbfbfe5cabeec19ad" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 44483 } } peers { permanent_uuid: "8de16c51296b48e29ec58c2f44e32c23" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 35005 } } }
I20260811 12:08:42.623826 13899 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8de16c51296b48e29ec58c2f44e32c23 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:08:42.626077 13575 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba6ae421929648f69827b8bd3621fbdc [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6ae421929648f69827b8bd3621fbdc" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40895 } } peers { permanent_uuid: "ad76e749a4b242ebbfbfe5cabeec19ad" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 44483 } } peers { permanent_uuid: "8de16c51296b48e29ec58c2f44e32c23" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 35005 } }
I20260811 12:08:42.628674 13575 leader_election.cc:288] T 00000000000000000000000000000000 P ba6ae421929648f69827b8bd3621fbdc [CANDIDATE]: Term 1 election: Requested vote from peers ad76e749a4b242ebbfbfe5cabeec19ad (127.9.221.189:44483), 8de16c51296b48e29ec58c2f44e32c23 (127.9.221.188:35005)
I20260811 12:08:42.630065 13826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ba6ae421929648f69827b8bd3621fbdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8de16c51296b48e29ec58c2f44e32c23"
I20260811 12:08:42.630764 13826 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8de16c51296b48e29ec58c2f44e32c23 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:08:42.635294 13826 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8de16c51296b48e29ec58c2f44e32c23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba6ae421929648f69827b8bd3621fbdc in term 1.
I20260811 12:08:42.636090 13533 leader_election.cc:302] T 00000000000000000000000000000000 P ba6ae421929648f69827b8bd3621fbdc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8de16c51296b48e29ec58c2f44e32c23, ba6ae421929648f69827b8bd3621fbdc; no voters:
I20260811 12:08:42.636662 13898 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba6ae421929648f69827b8bd3621fbdc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:08:42.648108 13898 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba6ae421929648f69827b8bd3621fbdc [term 1 LEADER]: Becoming Leader. State: Replica: ba6ae421929648f69827b8bd3621fbdc, State: Running, Role: LEADER
I20260811 12:08:42.648849 13898 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba6ae421929648f69827b8bd3621fbdc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6ae421929648f69827b8bd3621fbdc" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40895 } } peers { permanent_uuid: "ad76e749a4b242ebbfbfe5cabeec19ad" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 44483 } } peers { permanent_uuid: "8de16c51296b48e29ec58c2f44e32c23" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 35005 } }
I20260811 12:08:42.654698 13930 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba6ae421929648f69827b8bd3621fbdc [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba6ae421929648f69827b8bd3621fbdc. Latest consensus state: current_term: 1 leader_uuid: "ba6ae421929648f69827b8bd3621fbdc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6ae421929648f69827b8bd3621fbdc" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40895 } } peers { permanent_uuid: "ad76e749a4b242ebbfbfe5cabeec19ad" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 44483 } } peers { permanent_uuid: "8de16c51296b48e29ec58c2f44e32c23" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 35005 } } }
I20260811 12:08:42.655247 13930 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba6ae421929648f69827b8bd3621fbdc [sys.catalog]: This master's current role is: LEADER
I20260811 12:08:42.660292 13946 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:08:42.662369 13956 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 12:08:42.678432 13946 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:08:42.665949 13757 master_runner.cc:186] Error getting master registration for 127.9.221.190:40895: OK, instance_id { permanent_uuid: "ba6ae421929648f69827b8bd3621fbdc" instance_seqno: 1786450121443467 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 12:08:42.680181 13757 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 12:08:42.682526 13535 tablet.cc:2369] T 00000000000000000000000000000000 P ba6ae421929648f69827b8bd3621fbdc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:08:42.697419 13739 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.221.188" port: 35005 } attempt: 5
I20260811 12:08:42.702843 13753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:42.665423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54348 (local address 127.9.221.189:44483)
0811 12:08:42.665840 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:42.665847 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:42.677064 (+ 11217us) server_negotiation.cc:407] Connection header received
0811 12:08:42.677278 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:42.677287 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:42.677413 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:42.677540 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:42.682968 (+ 5428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:42.684325 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:42.686852 (+ 2527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:42.687724 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:42.687839 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:42.692595 (+ 4756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:42.692604 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:42.692613 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:42.692669 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:42.697244 (+ 4575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:42.698165 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:42.698171 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:42.698175 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:42.698329 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:42.698824 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:42.698828 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:42.698831 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:42.699195 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:42.699445 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:42.702194 (+ 2749us) server_negotiation.cc:299] Negotiation successful
0811 12:08:42.702466 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260}
Aug 11 12:08:42 dist-test-slave-mjjr krb5kdc[13475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450121, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.221.189@KRBTEST.COM for kudu/127.9.221.188@KRBTEST.COM
I20260811 12:08:42.708561 13826 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8de16c51296b48e29ec58c2f44e32c23 [term 1 FOLLOWER]: Refusing update from remote peer ba6ae421929648f69827b8bd3621fbdc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:08:42.709600 13898 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ba6ae421929648f69827b8bd3621fbdc [LEADER]: Connected to new peer: Peer: permanent_uuid: "8de16c51296b48e29ec58c2f44e32c23" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 35005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:08:42.703141 13495 master_runner.cc:186] Error getting master registration for 127.9.221.189:44483: OK, instance_id { permanent_uuid: "ad76e749a4b242ebbfbfe5cabeec19ad" instance_seqno: 1786450121972992 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 12:08:42.716198 13495 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 12:08:42.726027 13748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:42.698017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.221.188:35005 (local address 127.0.0.1:40364)
0811 12:08:42.698582 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:42.698604 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:08:42.698705 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:08:42.699594 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:08:42.699599 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:08:42.700101 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:08:42.700307 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:08:42.700318 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:08:42.701763 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:08:42.701768 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:08:42.703171 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:08:42.703187 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:08:42.704465 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:08:42.704472 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:08:42.704718 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:42.705290 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:08:42.705315 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:08:42.708900 (+ 3585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:08:42.718930 (+ 10030us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:08:42.718936 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:08:42.718942 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:08:42.720342 (+ 1400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:08:42.721393 (+ 1051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:08:42.721433 (+ 40us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:08:42.721438 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:08:42.721824 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:08:42.724080 (+ 2256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:08:42.724112 (+ 32us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:08:42.724521 (+ 409us) client_negotiation.cc:769] Sending connection context
0811 12:08:42.724780 (+ 259us) client_negotiation.cc:240] Negotiation successful
0811 12:08:42.725257 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450}
I20260811 12:08:42.726532 13892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:42.665205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45448 (local address 127.9.221.190:40895)
0811 12:08:42.665694 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:42.665702 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:42.689090 (+ 23388us) server_negotiation.cc:407] Connection header received
0811 12:08:42.689283 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:42.689288 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:42.689376 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:42.689479 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:42.692892 (+ 3413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:42.693990 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:42.699334 (+ 5344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:42.700096 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:42.700198 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:42.719333 (+ 19135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:42.719349 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:42.719354 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:42.719395 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:42.723180 (+ 3785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:42.724074 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:42.724080 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:42.724084 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:42.724224 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:42.724731 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:42.724736 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:42.724739 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:42.725092 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:42.725323 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:42.725906 (+ 583us) server_negotiation.cc:299] Negotiation successful
0811 12:08:42.726148 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302}
I20260811 12:08:42.734632 13887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:42.698924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40364 (local address 127.9.221.188:35005)
0811 12:08:42.699109 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:42.699115 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:42.699134 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:08:42.699185 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:42.699190 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:42.699284 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:42.699372 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:42.700477 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:42.701576 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:42.703379 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:42.704068 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:42.704191 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:42.709206 (+ 5015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:42.709212 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:42.709216 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:42.709265 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:42.716828 (+ 7563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:42.720710 (+ 3882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:42.720720 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:42.720726 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:42.721088 (+ 362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:42.722178 (+ 1090us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:42.722187 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:42.722192 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:42.723518 (+ 1326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:42.727350 (+ 3832us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:42.728277 (+ 927us) server_negotiation.cc:299] Negotiation successful
0811 12:08:42.729868 (+ 1591us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 12:08:42.734830 13899 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8de16c51296b48e29ec58c2f44e32c23 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba6ae421929648f69827b8bd3621fbdc. Latest consensus state: current_term: 1 leader_uuid: "ba6ae421929648f69827b8bd3621fbdc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6ae421929648f69827b8bd3621fbdc" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40895 } } peers { permanent_uuid: "ad76e749a4b242ebbfbfe5cabeec19ad" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 44483 } } peers { permanent_uuid: "8de16c51296b48e29ec58c2f44e32c23" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 35005 } } }
I20260811 12:08:42.735136 13899 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8de16c51296b48e29ec58c2f44e32c23 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:08:42.738931 13962 mvcc.cc:205] Tried to move back new op lower bound from 7317299702597947392 to 7317299702376763392. Current Snapshot: MvccSnapshot[applied={T|T < 7317299702597947392}]
I20260811 12:08:42.739928 13898 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba6ae421929648f69827b8bd3621fbdc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ba6ae421929648f69827b8bd3621fbdc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6ae421929648f69827b8bd3621fbdc" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40895 } } peers { permanent_uuid: "ad76e749a4b242ebbfbfe5cabeec19ad" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 44483 } } peers { permanent_uuid: "8de16c51296b48e29ec58c2f44e32c23" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 35005 } } }
I20260811 12:08:42.740240 13898 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba6ae421929648f69827b8bd3621fbdc [sys.catalog]: This master's current role is: LEADER
I20260811 12:08:42.740659 13739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad76e749a4b242ebbfbfe5cabeec19ad: Bootstrap starting.
I20260811 12:08:42.741816 13893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:42.666094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40362 (local address 127.9.221.188:35005)
0811 12:08:42.666306 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:42.666312 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:42.688245 (+ 21933us) server_negotiation.cc:407] Connection header received
0811 12:08:42.688524 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:42.688532 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:42.688647 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:42.688750 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:42.691140 (+ 2390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:42.692356 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:42.697764 (+ 5408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:42.698405 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:42.698529 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:42.712557 (+ 14028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:42.712564 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:42.712568 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:42.712610 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:42.718699 (+ 6089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:42.730874 (+ 12175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:42.730882 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:42.730887 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:42.731048 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:42.733813 (+ 2765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:42.733818 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:42.733820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:42.734204 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:42.734478 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:42.741016 (+ 6538us) server_negotiation.cc:299] Negotiation successful
0811 12:08:42.741355 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 12:08:42.747448 13739 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad76e749a4b242ebbfbfe5cabeec19ad: Neither blocks nor log segments found. Creating new log.
I20260811 12:08:42.749135 13739 log.cc:824] T 00000000000000000000000000000000 P ad76e749a4b242ebbfbfe5cabeec19ad: Log is configured to *not* fsync() on all Append() calls
I20260811 12:08:42.751577 13899 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8de16c51296b48e29ec58c2f44e32c23 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ba6ae421929648f69827b8bd3621fbdc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6ae421929648f69827b8bd3621fbdc" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40895 } } peers { permanent_uuid: "ad76e749a4b242ebbfbfe5cabeec19ad" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 44483 } } peers { permanent_uuid: "8de16c51296b48e29ec58c2f44e32c23" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 35005 } } }
I20260811 12:08:42.751854 13899 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8de16c51296b48e29ec58c2f44e32c23 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:08:42.752252 13946 catalog_manager.cc:1347] Generated new cluster ID: ce0c60e8e36e40b9a16f06e474b3b49d
I20260811 12:08:42.752349 13946 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:08:42.757570 13739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad76e749a4b242ebbfbfe5cabeec19ad: No bootstrap required, opened a new log
I20260811 12:08:42.769028 13739 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad76e749a4b242ebbfbfe5cabeec19ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6ae421929648f69827b8bd3621fbdc" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40895 } } peers { permanent_uuid: "ad76e749a4b242ebbfbfe5cabeec19ad" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 44483 } } peers { permanent_uuid: "8de16c51296b48e29ec58c2f44e32c23" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 35005 } }
I20260811 12:08:42.769503 13739 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad76e749a4b242ebbfbfe5cabeec19ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad76e749a4b242ebbfbfe5cabeec19ad, State: Initialized, Role: FOLLOWER
I20260811 12:08:42.770071 13739 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad76e749a4b242ebbfbfe5cabeec19ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6ae421929648f69827b8bd3621fbdc" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40895 } } peers { permanent_uuid: "ad76e749a4b242ebbfbfe5cabeec19ad" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 44483 } } peers { permanent_uuid: "8de16c51296b48e29ec58c2f44e32c23" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 35005 } }
I20260811 12:08:42.774572 13739 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad76e749a4b242ebbfbfe5cabeec19ad [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:08:42.774991 13696 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad76e749a4b242ebbfbfe5cabeec19ad [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:08:42.775480 13699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ba6ae421929648f69827b8bd3621fbdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ad76e749a4b242ebbfbfe5cabeec19ad"
I20260811 12:08:42.778693 13946 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:08:42.780607 13946 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:08:42.784945 13696 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ad76e749a4b242ebbfbfe5cabeec19ad [term 1 FOLLOWER]: Refusing update from remote peer ba6ae421929648f69827b8bd3621fbdc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:08:42.785992 13930 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ba6ae421929648f69827b8bd3621fbdc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad76e749a4b242ebbfbfe5cabeec19ad" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 44483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:08:42.788632 13699 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ad76e749a4b242ebbfbfe5cabeec19ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba6ae421929648f69827b8bd3621fbdc in term 1.
I20260811 12:08:42.774000 13970 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad76e749a4b242ebbfbfe5cabeec19ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6ae421929648f69827b8bd3621fbdc" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40895 } } peers { permanent_uuid: "ad76e749a4b242ebbfbfe5cabeec19ad" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 44483 } } peers { permanent_uuid: "8de16c51296b48e29ec58c2f44e32c23" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 35005 } } }
I20260811 12:08:42.791226 13970 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad76e749a4b242ebbfbfe5cabeec19ad [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:08:42.796150 13946 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ba6ae421929648f69827b8bd3621fbdc: Generated new TSK 0
I20260811 12:08:42.797127 13946 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:08:42.813329 13986 mvcc.cc:205] Tried to move back new op lower bound from 7317299702597947392 to 7317299702376763392. Current Snapshot: MvccSnapshot[applied={T|T < 7317299702597947392}]
I20260811 12:08:42.825038 13970 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad76e749a4b242ebbfbfe5cabeec19ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba6ae421929648f69827b8bd3621fbdc. Latest consensus state: current_term: 1 leader_uuid: "ba6ae421929648f69827b8bd3621fbdc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6ae421929648f69827b8bd3621fbdc" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40895 } } peers { permanent_uuid: "ad76e749a4b242ebbfbfe5cabeec19ad" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 44483 } } peers { permanent_uuid: "8de16c51296b48e29ec58c2f44e32c23" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 35005 } } }
I20260811 12:08:42.825424 13970 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad76e749a4b242ebbfbfe5cabeec19ad [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:08:42.825583 13982 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad76e749a4b242ebbfbfe5cabeec19ad [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ba6ae421929648f69827b8bd3621fbdc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6ae421929648f69827b8bd3621fbdc" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40895 } } peers { permanent_uuid: "ad76e749a4b242ebbfbfe5cabeec19ad" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 44483 } } peers { permanent_uuid: "8de16c51296b48e29ec58c2f44e32c23" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 35005 } } }
I20260811 12:08:42.825954 13982 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad76e749a4b242ebbfbfe5cabeec19ad [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:08:42.827095 13898 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba6ae421929648f69827b8bd3621fbdc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ba6ae421929648f69827b8bd3621fbdc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba6ae421929648f69827b8bd3621fbdc" member_type: VOTER last_known_addr { host: "127.9.221.190" port: 40895 } } peers { permanent_uuid: "ad76e749a4b242ebbfbfe5cabeec19ad" member_type: VOTER last_known_addr { host: "127.9.221.189" port: 44483 } } peers { permanent_uuid: "8de16c51296b48e29ec58c2f44e32c23" member_type: VOTER last_known_addr { host: "127.9.221.188" port: 35005 } } }
I20260811 12:08:42.827464 13898 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba6ae421929648f69827b8bd3621fbdc [sys.catalog]: This master's current role is: LEADER
I20260811 12:08:42.844954 13997 catalog_manager.cc:1259] Loaded cluster ID: ce0c60e8e36e40b9a16f06e474b3b49d
I20260811 12:08:42.845137 13997 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ad76e749a4b242ebbfbfe5cabeec19ad: loading cluster ID for follower catalog manager: success
I20260811 12:08:42.849659 13997 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ad76e749a4b242ebbfbfe5cabeec19ad: acquiring CA information for follower catalog manager: success
I20260811 12:08:42.852299 13997 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ad76e749a4b242ebbfbfe5cabeec19ad: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 12:08:42.907058 13892 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:08:42.895360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45458 (local address 127.9.221.190:40895)
0811 12:08:42.895578 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:42.895586 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:42.904370 (+ 8784us) server_negotiation.cc:407] Connection header received
0811 12:08:42.904736 (+ 366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:42.904746 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:42.904879 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:42.904975 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:42.906658 (+ 1683us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45458: BlockingRecv error: recv got EOF from 127.0.0.1:45458 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260811 12:08:42.908722 13753 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:08:42.895086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54358 (local address 127.9.221.189:44483)
0811 12:08:42.895291 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:42.895299 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:42.904780 (+ 9481us) server_negotiation.cc:407] Connection header received
0811 12:08:42.905107 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:42.905114 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:42.905235 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:42.905340 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:42.908486 (+ 3146us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54358: BlockingRecv error: recv got EOF from 127.0.0.1:54358 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260811 12:08:42.910593 13893 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:08:42.901207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40368 (local address 127.9.221.188:35005)
0811 12:08:42.901435 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:42.901444 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:42.908361 (+ 6917us) server_negotiation.cc:407] Connection header received
0811 12:08:42.908666 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:42.908673 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:42.908775 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:42.908865 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:42.910322 (+ 1457us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40368: BlockingRecv error: recv got EOF from 127.0.0.1:40368 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 12:08:42.931989 13753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:42.922931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54364 (local address 127.9.221.189:44483)
0811 12:08:42.923139 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:42.923147 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:42.923704 (+ 557us) server_negotiation.cc:407] Connection header received
0811 12:08:42.923989 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:42.924002 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:42.924243 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:42.924450 (+ 207us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:08:42.924969 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:42.926301 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:42.928396 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:42.929246 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:42.929410 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:42.930045 (+ 635us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:08:42.930883 (+ 838us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:08:42.931050 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:42.931198 (+ 148us) server_negotiation.cc:299] Negotiation successful
0811 12:08:42.931327 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260811 12:08:42.949376 13893 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:08:42.940558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40376 (local address 127.9.221.188:35005)
0811 12:08:42.940814 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:42.940822 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:42.947279 (+ 6457us) server_negotiation.cc:407] Connection header received
0811 12:08:42.947556 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:42.947565 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:42.947749 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:42.947831 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:42.949061 (+ 1230us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40376: BlockingRecv error: recv got EOF from 127.0.0.1:40376 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 12:08:42.969475 13753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:42.961654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54372 (local address 127.9.221.189:44483)
0811 12:08:42.961845 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:42.961851 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:42.962418 (+ 567us) server_negotiation.cc:407] Connection header received
0811 12:08:42.962614 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:42.962622 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:42.962755 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:42.962873 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:08:42.963346 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:42.964715 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:42.966919 (+ 2204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:42.967829 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:42.967986 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:42.968491 (+ 505us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:08:42.968676 (+ 185us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:08:42.968799 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:42.968984 (+ 185us) server_negotiation.cc:299] Negotiation successful
0811 12:08:42.969082 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260811 12:08:42.980700 13893 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:08:42.978242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40388 (local address 127.9.221.188:35005)
0811 12:08:42.978437 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:42.978444 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:42.978954 (+ 510us) server_negotiation.cc:407] Connection header received
0811 12:08:42.979245 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:42.979254 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:42.979361 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:42.979442 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:42.980464 (+ 1022us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40388: BlockingRecv error: recv got EOF from 127.0.0.1:40388 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 12:08:43.019814 13753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:43.012835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54386 (local address 127.9.221.189:44483)
0811 12:08:43.013036 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:43.013043 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:43.013179 (+ 136us) server_negotiation.cc:407] Connection header received
0811 12:08:43.013560 (+ 381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:43.013566 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:43.013656 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:43.013725 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:08:43.014488 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:43.015710 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:43.017659 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:43.018308 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:43.018447 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:43.018950 (+ 503us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:08:43.019098 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:08:43.019195 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:43.019372 (+ 177us) server_negotiation.cc:299] Negotiation successful
0811 12:08:43.019458 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260811 12:08:43.031870 13893 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:08:43.029578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40400 (local address 127.9.221.188:35005)
0811 12:08:43.029782 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:43.029788 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:43.030379 (+ 591us) server_negotiation.cc:407] Connection header received
0811 12:08:43.030542 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:43.030548 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:43.030646 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:43.030759 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:43.031607 (+ 848us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40400: BlockingRecv error: recv got EOF from 127.0.0.1:40400 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 12:08:43.050199 13753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:43.043125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54394 (local address 127.9.221.189:44483)
0811 12:08:43.043366 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:43.043372 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:43.043951 (+ 579us) server_negotiation.cc:407] Connection header received
0811 12:08:43.044155 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:43.044161 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:43.044256 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:43.044356 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:08:43.044858 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:43.046020 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:43.048058 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:43.048797 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:43.048921 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:43.049414 (+ 493us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:08:43.049555 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:08:43.049655 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:43.049821 (+ 166us) server_negotiation.cc:299] Negotiation successful
0811 12:08:43.049912 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
W20260811 12:08:43.059476 13893 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:08:43.057016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40402 (local address 127.9.221.188:35005)
0811 12:08:43.057335 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:43.057343 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:43.057938 (+ 595us) server_negotiation.cc:407] Connection header received
0811 12:08:43.058095 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:43.058101 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:43.058202 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:43.058323 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:43.059191 (+ 868us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40402: BlockingRecv error: recv got EOF from 127.0.0.1:40402 (error 108)
Metrics: {"server-negotiator.queue_time_us":174}
I20260811 12:08:43.078004 13753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:43.070704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54396 (local address 127.9.221.189:44483)
0811 12:08:43.070907 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:43.070913 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:43.071457 (+ 544us) server_negotiation.cc:407] Connection header received
0811 12:08:43.071681 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:43.071687 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:43.071795 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:43.071908 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:08:43.072374 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:43.073612 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:43.075613 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:43.076455 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:43.076589 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:43.077130 (+ 541us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:08:43.077286 (+ 156us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:08:43.077391 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:43.077556 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 12:08:43.077646 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260811 12:08:43.087776 13893 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:08:43.085113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40412 (local address 127.9.221.188:35005)
0811 12:08:43.085322 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:43.085328 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:43.085980 (+ 652us) server_negotiation.cc:407] Connection header received
0811 12:08:43.086280 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:43.086286 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:43.086396 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:43.086482 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:43.087519 (+ 1037us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40412: BlockingRecv error: recv got EOF from 127.0.0.1:40412 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 12:08:43.116180 13753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:43.108386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54400 (local address 127.9.221.189:44483)
0811 12:08:43.108594 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:43.108604 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:43.109155 (+ 551us) server_negotiation.cc:407] Connection header received
0811 12:08:43.109518 (+ 363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:43.109525 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:43.109627 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:43.109712 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:08:43.110465 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:43.111796 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:43.113992 (+ 2196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:43.114614 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:43.114742 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:43.115300 (+ 558us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:08:43.115482 (+ 182us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:08:43.115580 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:43.115798 (+ 218us) server_negotiation.cc:299] Negotiation successful
0811 12:08:43.115888 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260811 12:08:43.130095 13893 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:08:43.122583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40426 (local address 127.9.221.188:35005)
0811 12:08:43.122807 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:43.122813 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:43.128312 (+ 5499us) server_negotiation.cc:407] Connection header received
0811 12:08:43.128542 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:43.128549 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:43.128653 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:43.128776 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:43.129806 (+ 1030us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40426: BlockingRecv error: recv got EOF from 127.0.0.1:40426 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
I20260811 12:08:43.174108 13753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:43.166804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54414 (local address 127.9.221.189:44483)
0811 12:08:43.167035 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:43.167042 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:43.167606 (+ 564us) server_negotiation.cc:407] Connection header received
0811 12:08:43.167866 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:43.167874 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:43.167977 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:43.168085 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:08:43.168604 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:43.169786 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:43.171845 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:43.172573 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:43.172716 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:43.173236 (+ 520us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:08:43.173389 (+ 153us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:08:43.173485 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:43.173677 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 12:08:43.173770 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260811 12:08:43.190652 13893 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:08:43.184598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40428 (local address 127.9.221.188:35005)
0811 12:08:43.184793 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:43.184798 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:43.189128 (+ 4330us) server_negotiation.cc:407] Connection header received
0811 12:08:43.189292 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:43.189298 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:43.189403 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:43.189517 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:43.190406 (+ 889us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40428: BlockingRecv error: recv got EOF from 127.0.0.1:40428 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 12:08:43.264007 13753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:43.258204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54416 (local address 127.9.221.189:44483)
0811 12:08:43.258413 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:43.258419 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:43.258827 (+ 408us) server_negotiation.cc:407] Connection header received
0811 12:08:43.258973 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:43.258979 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:43.259081 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:43.259205 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:08:43.259579 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:43.260802 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:43.262067 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:43.262714 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:43.262866 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:43.263212 (+ 346us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:08:43.263380 (+ 168us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:08:43.263476 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:43.263580 (+ 104us) server_negotiation.cc:299] Negotiation successful
0811 12:08:43.263687 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260811 12:08:43.271968 13893 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:08:43.270073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40440 (local address 127.9.221.188:35005)
0811 12:08:43.270288 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:43.270294 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:43.270646 (+ 352us) server_negotiation.cc:407] Connection header received
0811 12:08:43.270803 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:43.270808 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:43.270913 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:43.271024 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:43.271732 (+ 708us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40440: BlockingRecv error: recv got EOF from 127.0.0.1:40440 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 12:08:43.408057 13753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:43.403210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54432 (local address 127.9.221.189:44483)
0811 12:08:43.403456 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:43.403462 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:43.403789 (+ 327us) server_negotiation.cc:407] Connection header received
0811 12:08:43.403953 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:43.403957 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:43.404026 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:43.404104 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:08:43.404468 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:43.405136 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:43.406424 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:43.406871 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:43.406961 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:43.407305 (+ 344us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:08:43.407442 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:08:43.407508 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:43.407702 (+ 194us) server_negotiation.cc:299] Negotiation successful
0811 12:08:43.407804 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260811 12:08:43.415309 13893 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:08:43.413530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40446 (local address 127.9.221.188:35005)
0811 12:08:43.413772 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:43.413778 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:43.414032 (+ 254us) server_negotiation.cc:407] Connection header received
0811 12:08:43.414161 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:43.414165 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:43.414238 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:43.414319 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:43.415052 (+ 733us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40446: BlockingRecv error: recv got EOF from 127.0.0.1:40446 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
I20260811 12:08:43.667079 13956 catalog_manager.cc:1259] Loaded cluster ID: ce0c60e8e36e40b9a16f06e474b3b49d
I20260811 12:08:43.667269 13956 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8de16c51296b48e29ec58c2f44e32c23: loading cluster ID for follower catalog manager: success
I20260811 12:08:43.669866 13956 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8de16c51296b48e29ec58c2f44e32c23: acquiring CA information for follower catalog manager: success
I20260811 12:08:43.671703 13956 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8de16c51296b48e29ec58c2f44e32c23: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 12:08:43.681105 13753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:43.675098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54442 (local address 127.9.221.189:44483)
0811 12:08:43.675304 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:43.675310 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:43.675700 (+ 390us) server_negotiation.cc:407] Connection header received
0811 12:08:43.675972 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:43.675976 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:43.676043 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:43.676124 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:08:43.676660 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:43.677334 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:43.679334 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:43.679793 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:43.679916 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:43.680470 (+ 554us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:08:43.680563 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:08:43.680628 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:43.680807 (+ 179us) server_negotiation.cc:299] Negotiation successful
0811 12:08:43.680867 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 12:08:43.693704 13893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:43.687216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40458 (local address 127.9.221.188:35005)
0811 12:08:43.687461 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:43.687478 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:43.687839 (+ 361us) server_negotiation.cc:407] Connection header received
0811 12:08:43.687961 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:43.687968 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:43.688160 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:43.688292 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:08:43.688659 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:43.689807 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:43.691095 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:43.691815 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:43.691948 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:43.692394 (+ 446us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:08:43.693096 (+ 702us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:08:43.693251 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:43.693299 (+ 48us) server_negotiation.cc:299] Negotiation successful
0811 12:08:43.693385 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 12:08:43.710148 13753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:43.699831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54450 (local address 127.9.221.189:44483)
0811 12:08:43.699998 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:43.700004 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:43.703459 (+ 3455us) server_negotiation.cc:407] Connection header received
0811 12:08:43.703777 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:43.703785 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:43.703883 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:43.703974 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:08:43.704672 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:43.705516 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:43.707770 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:43.708375 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:43.708473 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:43.709293 (+ 820us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:08:43.709425 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:08:43.709484 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:43.709708 (+ 224us) server_negotiation.cc:299] Negotiation successful
0811 12:08:43.709787 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 12:08:43.712292 14214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:43.699952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45472 (local address 127.9.221.190:40895)
0811 12:08:43.704587 (+ 4635us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:43.704593 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:43.704613 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:08:43.704721 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:43.704728 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:43.704947 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:43.705125 (+ 178us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:08:43.705670 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:43.706844 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:43.709244 (+ 2400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:43.709877 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:43.710014 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:43.710920 (+ 906us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:08:43.711322 (+ 402us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:08:43.711447 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:43.711699 (+ 252us) server_negotiation.cc:299] Negotiation successful
0811 12:08:43.711959 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4513,"thread_start_us":4400,"threads_started":1}
I20260811 12:08:43.713447 13893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:43.699433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40466 (local address 127.9.221.188:35005)
0811 12:08:43.699598 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:43.699604 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:43.704212 (+ 4608us) server_negotiation.cc:407] Connection header received
0811 12:08:43.705509 (+ 1297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:43.705515 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:43.705600 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:43.705660 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:08:43.706604 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:43.707867 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:43.710235 (+ 2368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:43.710911 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:43.711023 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:43.712604 (+ 1581us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:08:43.712740 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:08:43.712843 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:43.713078 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 12:08:43.713163 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 12:08:43.716944 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 13495
I20260811 12:08:43.718078 13608 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:08:43.835055 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 13495
I20260811 12:08:43.874027 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 13625
W20260811 12:08:43.878310 13652 connection.cc:535] server connection from 127.0.0.1:54324 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 12:08:43.878587 13732 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:08:44.007715 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 13625
I20260811 12:08:44.032876 10102 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu pid 13757
I20260811 12:08:44.034117 13870 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:08:44.152082 10102 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGvgjmQ/build/asan/bin/kudu with pid 13757
2026-08-11T12:08:44Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4827 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9783 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (380969 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST